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© Sound-image position control apparatus. 



© In order to obtain a sound-broadened image and 
a clear sound-image discrimination image when pro- 
ducing plural kinds of sounds, the electronic musical 
instrument and the like provides a sound-image po- 
sition control apparatus. This apparatus at least pro- 
vides a signal mixing portion (e.g., matrix controller; 
MTR1) and a virtual-speaker position control portion 
(DL10-DL13. KL10-KL13, KR10-KR13, AD10-AD13). 
Herein, the signal mixing portion mixes plural audio 
signals supplied from a sound source (17) and the 
<§^5 like in accordance with a predetermined signal mix- 
^ ing procedure so as to output plural mixed signals. 
^ In order to control positions of virtual speakers 
^ (VS10-VS13) which are emerged as sound-produc- 
ts ing points as if each kind of sounds is produced 
from each of these points, the virtual-speaker posi- 
tion control portion applies different delay times to 

m 



each of plural mixed signals so as to output delayed 
signals as right-side and left-side audio signals to be 
respectively supplied to right-side and left-side 
speakers (SP(R), SP(L)). Thus, the sound-image po- 
sitions formed by the virtual speakers are controlled 
well, so that the person can clearly discriminate and 
recognize each of the sound-image positions. When 
applying this apparatus to the game device providing 
a display unit which displays an animated image 
representing a visual image of the air plane and the 
like, by adequately controlling the sound-image posi- 
tion, it is possible to obtain a brand-new live-audio 
effect, by which the point of producing the sounds 
corresponding to the animated image can be moved 
in accordance with the movement of the animated 
image which is moved by the player of the game. 
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The present invention relates to a sound-image 
position control apparatus which is suitable for use 
in the electronic musical instruments, audio-visual 
devices and the like so as to eventually perform 
the sound-image localization. 

As the device which offers the person the 
sound-broadened image, there are provided the 
stereo-chorus device, reverberation device and the 
like. Herein, the former one is designed to produce 
the sound of which phase is slightly shifted as 
compared to that of the original sound so that this 
phase-shifted sound and the original sound are 
alternatively produced from the left and right loud- 
speakers, while the latter one is designed to impart 
the reverberation effect to the sounds. 

In addition, there is another device, called the 
panning device. This panning device is designed to 
provide the predetermined output-level difference 
between the sounds which are respectively pro- 
duced from the left and right loud-speakers, result- 
ing that the stereophonic effect or stereo-impres- 
sive image is applied to the sounds. 

The above-mentioned stereo-chorus device or 
reverberation device can enlarge the sound-broad- 
ened image. However, there is a drawback in that 
the sound-distribution image which is sensed by 
the listener must become unclear when enlarging 
the sound-broadened image. Herein, the sound- 
distribution image is defined as a degree of dis- 
crimination in which the person who listens to the 
music from the audio device can specifically dis- 
criminate the sound of certain instrument from the 
other sounds. For example, when listening to the 
music played by the guitar and keyboard by the 
audio device having a relatively good sound-dis- 
tribution image control, the person can discriminate 
the respective sounds as if the guitar sound is 
produced from the predetermined left-side position, 
while the keyboard sound is produced from the 
predetermined right-side position (hereinafter, such 
virtual position will be referred to as the sound- 
image position). When listening to the music by 
use of the aforementioned stereo-chorus device or 
reverberation device, it is difficult for the person to 
clearly discriminate the sound-image positions. 

In the panning device, the sound-image posi- 
tion must be fixed at the predetermined position 
disposed on the line connecting the left and right 
loud-speakers on the basis of the sound-image 
localization technique, resulting that the sound- 
broadened image cannot be substantially obtained. 
In other words, when simultaneously producing plu- 
ral sounds each having a different sound-image 
position, the panning device merely functions to 
roughly mix up those sounds so that the clear 
sound-image positions cannot be obtained. 

In the meantime, the panning device is fre- 
quently equipped with or built in the electronic 



musical instrument when simulating the sounds of 
the relatively large-scale instruments such as the 
piano, organ and vibraphone. In such instrument 
(e.g., piano), the sound-producing positions must 
5 be moved accompanied with the progression of 
notes, thus, the panning device functions to simu- 
late such movement of the sound-producing posi- 
tions. 

However, the panning device also suffers from 

10 the aforementioned drawback. More specifically, 
the panning device can offer certain degree of 
panning effect when simulating the sounds, how- 
ever, it is not possible to clearly discriminate the 
sound-image position of each of the sounds to be 

15 produced. In short, the panning device cannot per- 
form the accurate simulation with respect to the 
discrimination of the sound-image positions. 

It is accordingly a primary object of the present 
invention to provide a sound-image position control 

20 apparatus by which even when simultaneously pro- 
ducing plural sounds each having a different 
sound-image position, it is possible to clearly dis- 
criminate the sound-image position of each of the 
sounds to be produced. 

25 It is another object of the present invention to 

provide a sound-image position control apparatus 
which can offer the sound-broadened effect, stereo- 
phonic effect or stereo-impressive image when si- 
multaneously producing plural sounds each having 

30 a different sound-image position. 

It is a further object of the present invention to 
provide a sound-image position control apparatus 
which can offer a sound-image localization with a 
simple configuration of the apparatus. 

35 According to the fundamental configuration of 

the present invention, the sound-image position 
control apparatus comprises a signal mixing portion 
and a virtual-speaker position control portion. Here- 
in, the signal mixing portion mixes plural audio 

40 signals supplied thereto in accordance with a pre- 
determined signal mixing procedure so as to output 
plural mixed signals. The virtual-speaker position 
control portion applies different delay times to each 
of plural mixed signals so as to output delayed 

45 signals as right-side and left-side audio signals to 
be respectively supplied to right-side and left-side 
speakers. In this case, some virtual speakers are 
virtually emerged as sound-producing points as if 
each of the sounds is produced from each of these 

so points. Thus, sound-image positions formed by the 
virtual speakers are controlled in accordance with 
plural mixed signals. 

Under effect of the aforementioned configura- 
tion of the present invention, the sounds applied 

55 with the stereophonic effect and clear sound-image 
discrimination effect are to be actually produced 
from the right-side and left-side speakers as if the 
sounds are virtually produced from the virtual 
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speakers of which positions are determined under 
control of the virtual-speaker position control por- 
tion. 

When applying this apparatus to the game 
device providing a display unit which displays an 
animated image representing an image of the air 
plane and the like, by adequately controlling the 
sound-image position, it is possible to obtain a 
brand-new live-audio effect, by which the point of 
producing the sounds corresponding to this ani- 
mated image is moved in accordance with the 
movement of the animated image which is moved 
by the player of the game. 

Moreover, the present invention can be easily 
modified to be applied to the movie system or 
video game device in which the sound-image posi- 
tion is controlled responsive to the video image. 
This system comprises an audio/video signal pro- 
ducing portion; a scene-identification signal produc- 
ing portion; a plurality of speakers; a sound-image 
forming portion; and a control portion. 

The above-mentioned scene-identification sig- 
nal producing portion outputs a scene-identification 
signal in response to a scene represented by the 
video signal. The sound-image forming portion per- 
forms the predetermined processings on the audio 
signals so as to drive the speakers. Under effect of 
such signal processings, the speakers produce the 
sounds of which sound-image positions are fixed at 
the desirable positions departing from the linear 
spaces directly connecting the speakers. The con- 
trol portion controls the contents of the signal pro- 
cessings so as to change over the fixed sound- 
image position in response to the scene-identifica- 
tion signal. 

Further objects and advantages of the present 
invention will be apparent from the following de- 
scription, reference being had to the accompanying 
drawings wherein the preferred embodiments of 
the present invention are clearly shown. 

In the drawings: 

Fig. 1(A) is a block diagram showing an elec- 
tronic configuration of a sound-image position 
control apparatus according to a first embodi- 
ment of the present invention; 
Fig. 1(B) is a plan view illustrating a position 
relationship between the performer and speak- 
ers; 

Fig. 2(A) is a block diagram showing another 
example of the arrangement of circuit elements 
in a matrix controller; 

Fig. 2(B) is a plan view illustrating another ex- 
ample of the position relationship between the 
performer and speakers; 

Fig. 3(A) is a block diagram showing a detailed 
electronic configuration of a cross-talk canceler 
shown in Fig. 1(A); 



Fig. 3(B) is a plan view illustrating another ex- 
ample of the position relationship between the 
performer and speakers; 

Fig. 4 is a plan view illustrating a fundamental 
s position relationship between the performer and 

speakers according to the present invention; 

Fig. 5 is a block diagram showing a modified 

example of the first embodiment; 

Fig. 6 is a block diagram showing an electronic 
70 configuration of a sound-image position control 

apparatus according to a second embodiment of 

the present invention; 

Fig. 7 is a drawing showing a relationship be- 
tween the person and virtual sound source; 

75 Fig. 8 is a block diagram showing an electronic 
configuration of a game device to which a 
sound-image position control apparatus accord- 
ing to a third embodiment of the present inven- 
tion is applied; 

20 Fig. 9 is a drawing showing a two-dimensional 
memory map of a coordinate/sound-image-posi- 
tion coefficient conversion memory shown in 
Fig. 8; 

Fig. 10 is a plan view illustrating a position 
25 relationship between the player and game de- 
vice; 

Fig. 1 1 is a block diagram showing an electronic 

configuration of a video game system; 

Fig. 12 is a block diagram showing an electronic 

30 configuration of a sound-image position control 
apparatus, shown in Fig. 11, according to a 
fourth embodiment of the present invention; 
Fig. 13 is a drawing illustrating a position rela- 
tionship among a listener, loud-speakers and a 

35 video screen; 

Fig. 14 illustrates a polar-coordinate system 
which is used for defining a three-dimensional 
space; and 

Fig. 15 is a block diagram showing a typical 
40 example of a virtual-speaker system, of which 
concept is applied to the fourth embodiment. 
Now, description will be given with respect to 
the embodiments of the present invention by refer- 
ring to the drawings, wherein the predetermined 
45 position relationship is fixed between a performer P 
and an instrument I as shown in Fig. 4. In the 
description, the lateral direction indicates an arrow 
direction "a", while the longitudinal direction in- 
dicates an arrow direction "b" as shown in Fig. 4. 

50 

[A] First Embodiment 

(1) Configuration 

55 Fig. 1(B) is a plan view illustrating a position 

relationship between a person M (i.e., performer) 
and an electronic musical instrument containing 
two speakers (i.e., loud-speakers). Herein, KB des- 
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ignates a keyboard providing plural keys, wherein 
when depressing a key, a tone generator (not 
shown) produces a musical tone waveform signal 
having the pitch corresponding to the depressed 
key. SP(L) and SP(R) designate left and right 
speakers respectively. These speakers SP(L), SP- 
(R) are respectively arranged at the predetermined 
left-side and right-side positions of the upper por- 
tion of the instrument 

Fig. 1(A) is a block diagram showing an elec- 
tronic configuration of a sound-image position con- 
trol apparatus 1 according to a first embodiment of 
the present invention. This apparatus 1 provides 
eight channels respectively denoted by numerals 
Ch10 to Ch17 (given with a general numeral "Ch"), 
wherein each channel Ch receives the musical tone 
waveform signal produced from the tone generator. 
Specifically, the musical tone waveform signal sup- 
plied to each channel Ch has the allocated fre- 
quency domain corresponding to some musical 
notes (hereinafter, referred to as the allocated tone 
area). 

More specifically, the allocation of the tone 
areas is given as follows: the musical tone 
waveform signal of which tone area corresponds to 
the lowest-pitch note to C1 note is supplied to the 
channel Ch10, while the musical tone waveform 
signal of which tone area corresponds to C#1 note 
to C2 note is supplied to the channel Ch11. Simi- 
larly, the tone area of C#2 to F2 is allocated to the 
channel Ch12; the tone area of F#2 to C3 is al- 
located to the channel Ch13; the tone area of C#3 
to F3 is allocated to the channel Ch14; the tone 
area of F#3 to C4 is allocated to the channel Ch1 5; 
the tone area of C#4 to C#5 is allocated to the 
channel Ch16; and the tone area corresponding to 
the D5 note to the highest-pitch note is allocated to 
the channel Ch17. 

Next, M1 to M12 designate multipliers which 
multiply the musical tone waveform signal supplied 
thereto by respective coefficients CM1 to CM12. 
IN10 to IN13 designate adders, each of which 
receives the outputs of some multipliers. The 
above-mentioned elements, i.e., multipliers M1 to 
M12, adders IN10 to IN13 and channels Ch10 to 
Ch17 are assembled together into a matrix control- 
ler MTR1. Herein, the connection relationship and 
arrangement relationship among those elements of 
the matrix controller MTR1 can be arbitrarily 
changed in response to the control signal and the 
like. Incidentally, the detailed explanation of this 
matrix controller MTR1 will be given later. 

Meanwhile, DL10 to DL13 designate delay cir- 
cuits which respectively delays the outputs of the 
adders IN10 to IN13. Each of them has two output 
terminals each having a different delay time. 

The signal outputted from a first output termi- 
nal TL10 of the delay circuit DUO is multiplied by 
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the predetermined coefficient by a multiplier KL10, 
and then the multiplied signal is supplied to a first 
input (i.e., input for the left-side speaker) of a 
cross-talk canceler 2 via an adder AD10. On the 

5 other hand, the signal outputted from a second 
output terminal TR10 of the delay circuit DL10 is 
multiplied by the predetermined coefficient by a 
multiplier KR10, and then the multiplied signal is 
supplied to a second input (i.e., input for the right- 

w side speaker) of the cross-talk canceler 2 via ad- 
ders AD12, AD13. 

Similarly, the signal outputted from a first ter- 
minal TL1 1 of the delay circuit DL1 1 is eventually 
supplied to the first input of the cross-talk canceler 

75 2 via a multiplier KL11 and the adder AD10, while 
another signal outputted from a second terminal 
TR11 of the delay circuit DL11 is eventually sup- 
plied to the second input of the cross-talk canceler 
2 via a multiplier KR11 and the adders AD12, 

20 AD13. The signal outputted from a first terminal 
TL12 of the delay circuit DL12 is eventually sup- 
plied to the first input of the cross-talk canceler 2 
via a multiplier KL12 and the adder AD11, AD10, 
while another signal outputted from a second termi- 

25 nal TR12 of the delay circuit DL12 is eventually 
supplied to the second input of the cross-talk can- 
celer 2 via a multiplier KR12 and the adder AD13. 
Lastly, the signal outputted from a first terminal 
TL13 of the delay circuit DL13 is eventually sup- 

30 plied to the first input of the cross-talk canceler 2 
via a multiplier KL13 and the adders AD11, AD10, 
while another signal outputted from a second termi- 
nal TR13 of the delay circuit DL13 is eventually 
supplied to the second input of the cross-talk can- 

35 celer 2 via a multiplier KL13 and the adder AD13. 

The above-mentioned cross-talk canceler 2 is 
designed to cancel the cross-talk sounds which are 
emerged when the person hears the sounds with 
his both ears. In other words, this is designed to 

40 eliminate the cross-talk phenomenon in which the 
right-side sound is entered into the left ear, while 
the left-side sound is entered into the right ear. Fig. 
3(A) shows an example of the circuitry of this 
cross-talk canceler 2. This circuit is designed on 

45 the basis of the transfer function of head which is 
obtained through the study of the sound transmis- 
sion between the human ears and dummy head 
(i.e., virtual simulation model of the human head). 
On the basis of the experimental values obtained 

50 through the transfer function of head, the study is 
made to compute the sound-arrival time differences 
between the left and right ears and the peak values 
of the impulse response of the transfer function. In 
response to these values, this circuitry performs 

55 the delay operations and weight functional 
caluculus. 

The observation is made on the model wherein 
both of the speakers SP(L), SP(R) are positioned 



5 



* 



7 EPO 



apart from the person M by 1 -5 m respectively and 
they are also respectively arranged at the predeter- 
mined left-side and right-side positions each of 
which direction is deviated from the front direction 
of the person M by 45" . Since the foregoing 
transfer function of head of the person M is the 
symmetrical function, one of the speaker SP(L), 
SP(R) is sounded so as to actually measure the 
sound-arrival time difference between the left and 
right ears and the peak values of the impulse 
response. Herein, coefficients of multipliers and 
delay times of delay circuits in the circuitry shown 
in Fig. 3(A) are determined on the basis of the 
result of the measurement. For example, when the 
result of the measurement indicates that the 
left/right level difference is at 6dB (or 0.5) and the 
left/right time difference is at 200 u s, the same 
coefficient "-0.5" is applied to multipliers KL30, 
KR32, while the same delay time 200 u s is set to 
delay circuits DL30, DL32. Incidentally, the other 
circuit elements in Fig. 3(A), i.e. , delay circuits 
DL31, DL33 and multipliers KL31, KR33 configure 
the all-pass filter which is provided to perform the 
phase matching. 

As shown in Fig. 1(A), the left and right output 
signals of the cross-talk canceler 2 are amplified by 
an amplifier 3 and then supplied to the left and 
right speakers SP(L), SP(R), from which the cor- 
responding left/right sounds are produced. When 
listening to the sounds which are produced by 
means of the cross-talk canceler 2, the cross talk is 
canceled, resulting that the clear sound separation 
between the left/right speakers is achieved. 

Next, the description will be given with respect 
to the functions of the delay circuits DL10-DL13. In 
case of the delay circuit DL10, the signal outputted 
from the terminal TR10 is multiplied by the pre- 
determined coefficient in the multiplier KR10, and 
consequently, the multiplied signal will be con- 
verted into the musical sound by the right speaker 
SP(R). On the other hand, the signal outputted from 
the terminal TL10 is multiplied by the predeter- 
mined coefficient in the multiplier KL10, and con- 
sequently, the multiplied signal will be converted 
into the musical sound by the left speaker SP(L). In 
this case, the sound-image position is determined 
by two factors, i.e., the difference between the 
delay times of the sounds respectively produced 
from the right and left speakers, and the ratio 
between the tone volumes respectively applied to 
the left and right speakers. Since the present em- 
bodiment can set the above-mentioned delay-time 
difference in addition to the above-mentioned tone- 
volume ratio, the sound-image position can be set 
at certain position which is far from the speakers 
SP(L), SP(R) and which departs from the line con- 
necting these speakers. In short, it is possible to 
set the sound-image position in the arbitrary space 
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which departs from the linear space connecting the 
speakers. In other words, the virtual speakers 
which are not actually existed are placed at the 
arbitrary spatial positions, so that the person can 

5 listen to the sounds which are virtually produced 
from those positions. In the present embodiment, 
the delay circuit DL10 functions to set the virtual 
sound-producing position at VS10 (see Fig. 1(B)), 
which is called as the virtual speaker. 

w Similarly, the other delay circuits DL11, DL12, 

DL13 respectively correspond to the virtual speak- 
ers VS11, VS12, VS13 as shown in Fig. 1(B). As 
shown in Fig. 1(B), these virtual speakers VS10, 
VS11, VS12, VS13 are respectively and roughly 

is arranged along with a circular line which can be 
drawn about the performer. When drawing the cen- 
ter line between the performer (i.e., circle center) 
and respective one of the virtual speakers VS10, 
VS11, VS12, VS13, there are formed four circular 

20 angles, 60* ,24° ,24* ,60° as shown in Fig. 1- 
(B). 

Next, the description will be given with respect 
to the functions of the matrix controller MTR1. As 
described before, this matrix controller MTR1 is 

25 designed to control the connection relationship and 
arrangement relationship among the multipliers M1- 
M12, adders IN10-IN13 and channels Ch10-Ch17. 
Such control indicates how to assign the signals of 
the channels Ch10-Ch17 to the virtual speakers 

30 VS10-VS13. Thus, the sound-image position of 
each channel Ch can be determined by the ratio of 
each channel-output signal applied to each virtual 
speaker. In other words, the panning control is 
carried out on the virtual speakers VS10-VS13 re- 

35 spectively, thus controlling the sound-image posi- 
tion with respect to each channel. 

In the present embodiment as shown in Fig. 1- 
(A), the allocation ratio of the each channel-output 
signal applied to each virtual speaker is controlled 

40 by setting the coefficients of the multipliers M1- 
M12 as follows: CM1 =0.75 (by being multiplied by 
this coefficient, the tone volume of the musical tone 
waveform signal is reduced by 2.5dB), CM2 = 0.75, 
CM3 = 0.25 (by being multiplied by this coefficient, 

45 the tone volume of the musical tone waveform 
signal is reduced by 12dB), CM4 = 0.75, 
CM5 = 0.625 (by being multiplied by this coeffi- 
cient, the tone volume of the musical tone 
waveform signal is reduced by 4.08dB), 

50 CM6 = 0.313 (which is equivalent to the reduction of 
10.08dB in the tone volume of the musical tone 
waveform signal), CM7 = 0.313, CM8 = 0.625, 
CM9 = 0.75, CM10 = 0.25, CM11=0.75, 

CM12 = 0.75. 

55 Fig. 2(A) shows another example of the ar- 

rangement and connection among the multipliers 
and adders under control of the matrix controller 
MTR1. In this example, only two delay circuits 
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DL10, DL13 are used for the virtual speakers. In 
short, as shown in Fig. 2(B), two virtual speakers 
VS10, VS13 are used for the production of the 
musical sounds. Herein, under control of the matrix 
controller MTR1 , some of the signals of the chan- 
nels Ch10-Ch17 are adequately allocated to each 
of the adders IN10, IN13 so as to control the 
sound-image positions. In this example, the coeffi- 
cients of the multipliers M1-M14 are respectively 
set as follows: CM1=0.75, CM2 = 0.75, 
CM3 = 0.313, CM4 = 0.625, CM5 = 0.375 (by being 
multiplied by this coefficient, the tone volume of 
the musical tone waveform signal is reduced by 
8.5dB), CM6 = 0.5 (which is equivalent to the reduc- 
tion of 6dB in the tone volume of the musical tone 
waveform signal), CM7 = 0.439 (which is equivalent 
to the reduction of 7.1 6dB in the tone volume of 
the musical tone waveform signal), CM8 = 0.439, 
CM9 = 0.5, CM1 0 = 0.375, CM1 1 = 0.625, 
CM12 = 0.313, CM13 = 0.75, CM14 = 0.75. 

(2) Operation 

Next, the description will be given with respect 
to the operation of the present embodiment. 

When the performer P plays the keyboard to 
perform the music, the musical tone waveform sig- 
nal is produced in response to each of the keys 
depressed by the performer. Then, the musical 
tone waveform signals are respectively allocated to 
the channels on the basis of the predetermined 
tone-area allocation manner, so that these signals 
are eventually entered into the matrix controller 
MTR1. Assuming that the circuit elements of the 
matrix controller MTR1 are arranged and connect- 
ed as shown in Fig. 1(A), the musical tone 
waveform signals are produced as the musical 
sounds from the virtual speakers VS10-VS13 in 
accordance with their tone areas. 

The detailed explanation can be described as 
follows. First of all, the musical tone waveform 
signals corresponding to the tone area between the 
lowest-pitch note and C1 note (see Ch10) are pro- 
duced as the musical sounds from the virtual 
speaker VS10. In addition, the musical tone 
waveform signals corresponding to the tone area 
between the C#1 note and C2 note (see Ch11) are 
produced as the musical sounds from the virtual 
speakers VS12, VS10. However, due to the coeffi- 
cients of the multipliers M2, M3, the sound-image 
positions corresponding to those notes are placed 
close to the virtual speaker VS10. More specifi- 
cally, these sound-image positions are arranged on 
the line connecting the virtual speakers VS12, 
VS10, but they are also located close to the virtual 
speaker VS10. Further, the musical tone waveform 
signals corresponding to the tone area between the 
C#2 note to F2 note (see Ch12) are produced as 



the musical sounds from the virtual speaker VS11. 
Similarly, the other musical tone waveform signals 
corresponding to each the other tone areas (i.e., 
each of the other channels) are eventually pro- 
5 duced as the musical sounds from the predeter- 
mined one or two virtual speakers at certain sound- 
image positions. Thus, the sound-image positions 
corresponding to the tone areas which are respec- 
tively arranged from the lowest pitch to the highest 

10 pitch are sequentially arranged from the left-side 
position to the right-side position along with a cir- 
cular line drawn about the performer P (see Fig. 1- 
(B)). As a result, when the performer P sequentially 
depress the keys from the lower pitch to the higher 

rs pitch, the sound-image positions are sequentially 
moved from the left-side position to the right-side 
position along with the above-mentioned circular 
line. In short, it is possible to control the left/right 
and front/back positionings of the sound images. 

20 On the other hand, when the circuit elements 

of the matrix controller MTR1 are arranged and 
connected as shown in Fig. 2(A), the musical tone 
waveform signals of each tone area are eventually 
produced as the musical sounds from one or both 

25 of the virtual speakers VS10, VS13. Thus, the posi- 
tioning control of the sound images are controlled 
on the line connecting these virtual speakers. In 
this case, the control of the front/back-side sound- 
broadened image is poor as compared to that of 

30 Fig. 1(A). However, as comparing to the state 
where the musical sounds are merely produced 
from the left/right speakers SP(L), SP(R), this ex- 
ample can improve the control of the front/back- 
side sound broadened image. 

35 As described heretofore, the first embodiment 

is designed to change the allocation manner of the 
musical tone waveform signals by use of the matrix 
controller MTR1 , therefore, it is possible to change 
over the control manner of the sound images with 

40 ease. 

(3) Modified Example 

Fig. 5 is a block diagram showing a modified 
45 example of the foregoing first embodiment, in 
which there are provided eight delay circuits DL50- 
DL57 used for emerging the virtual speakers. In 
Fig. 5, the illustration is partially omitted, so that 
there are also provided eight adders, in the matrix 
50 controller MTR1, respectively corresponding to the 
above-mentioned eight delay circuits DL50-DL57. 
According to the configuration of this modified ex- 
ample, eight virtual speakers are emerged, so that 
the musical tone waveform signals can be ade- 
55 quately allocated to these virtual speakers. Due to 
the provision of eight virtual speakers, it is possible 
to perform the more precisely control on the 
sound-image positions. 
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[B] Second Embodiment 

Next, description will be given with respect to 
the second embodiment of the present invention by 
referring to Fig. 6, wherein some parts correspond- 5 
ing to those of the foregoing first embodiment are 
omitted. 

In Fig. 6, numerals STR60-STR65 designate 
respective tone generators which are controlled by 
the MIDI signal (i.e., digital signal of which format is 70 
based on the standard for Musical Instruments 
Digital Interface). In short, one of the tone gener- 
ators STR60-STR65 designated by the MIDI signal 
is activated to produce the musical tone waveform 
signal. The outputs of these tone generators 75 
STR60-STR65 are respectively supplied to the de- 
lay circuits DL60-DL65 which are used for forming 
the virtual speakers respectively. Then, the outputs 
of the delay circuits DL60-DL65 are multiplied by 
the predetermined coefficients respectively, so that 20 
some of the multiplied outputs are added together 
in adders VSR1-VSR4, VSL1-VSL4, of which addi- 
tion results are supplied to the cross-talk canceler 
2. 

According to the configuration of the above- 25 
mentioned second embodiment, the output of each 
tone generator is produced as the musical sound 
from certain virtual speaker. Thus, when respec- 
tively connecting six strings of the guitar with six 
tone generators STR60-STR65, it is possible to 30 
well simulate the sound-producing manner of the 
guitar with respect to each string. The reason why 
such well simulation can be performed by the 
second embodiment is as follows: 

When the guitar is located close to the listener 35 
so that the strings are also located close to the 
ears of the listener, the listener can clearly dis- 
criminate the separate sound produced from each 
string of the guitar. However, as the distance be- 
tween the listener and guitar becomes larger, the 40 
sound-separation image of each string of the guitar 
becomes weaker. Therefore, in the end, the sounds 
produced from all strings of the guitar will be heard 
as one overall sounds which are produced from 
one sound-production point. Thus, by adequately 45 
setting the delay times of the delay circuits DL60- 
DL65 and the coefficients which are multiplied with 
the outputs of the delay circuits DL60-DL65, it is 
possible to offer the image of the distance by 
which the instrument is departed from the listener. 50 

In the meantime, it is possible to compute the 
distance between the person and the virtual sound 
source which is embodied by the delay circuit as 
shown in Fig. 7. Herein, "r" designates a radius of 
the head of the person M; "d" designates a dis- 55 
tance between the sound source and the center of 
head; and "0" designates an angle which is formed 
between the sound source and the front-direction 
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line of the head. In this case, it is possible to 
compute distances "dr" and "dl" by the following 
equations, wherein "dr" designates a distance be- 
tween the sound source and the right ear of the 
person, while "dl" designates a distance between 
the sound source and the left ear of the person. 

dr 2 = r 2 + d 2 -2rd* sine (1) 

dl 2 = r 2 -d 2 + 2rd* sine (2) 

Thus, by computing these distances dr, dl with 
respect to each of the strings, it is possible to 
determine the factors for designing the delay cir- 
cuits DL60-DL65 respectively. 

Incidentally, in the aforementioned embodi- 
ments, it is possible for the user to arbitrarily set 
the connection pattern of the matrix controller 
MTR1 and the coefficient applied to each of the 
multipliers. Or, it is possible to store plural connec- 
tion patterns and plural values for each coefficient 
in advance, so that the user can arbitrarily select 
one of them. 

[C] Third Embodiment 

Next, description will be given with respect to 
the third embodiment of the present invention, in 
which the sound-image position control apparatus 1 
is applied to a game device 9, by referring to Figs. 
8 to 10. 

Fig. 8 is a block diagram showing an electronic 
configuration of a game device 9. Herein, 10 des- 
ignates a controller which controls the joy-stick 
unit, tracking-ball unit and several kinds of push- 
button switches (not shown) so that the operating 
states of them are sent to a control portion 1 1 . The 
control portion 11 contains a central processing 
unit (i.e., CPU) and several kinds of interface cir- 
cuits, whereas it is designed to execute the pre- 
determined game programs stored in a program 
memory 12. Thus, the game is in progress, while 
the overall control of the game device is performed 
by the control portion 11. In the progress of the 
game, a working memory 13 is collecting and 
storing several kinds of data which are obtained 
through the execution of the game programs. In 
response to the game program to be executed, a 
visual image information memory 14 stores visual 
image data to be displayed, representing the in- 
formation of the visual images corresponding to 
character images C1 , C2, C3 (given with the gen- 
eral numeral "C") and background images BG1, 
BG2, BG3 (given with the general numeral "BG"). 
These character images may correspond to the 
visual images of person, automobile, air plane, ani- 
mal, or other kinds of objects. The above-men- 
tioned visual image data are read out in the 
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progress of the game, so that the corresponding 
visual image is displayed at the predetermined 
position of a display screen of a display unit 15 by 
the predetermined display size in response to the 
progress of the game. 

Next, a coordinate/sound-image-position coeffi- 
cient conversion memory 16 stores parameters by 
which the display position of the character C in the 
display unit 15 is located at the proper position 
corresponding to the sound-image position in the 
two-dimensional area. Fig. 9 shows a memory con- 
figuration of the above-mentioned 
coordinate/sound-image-position coefficient conver- 
sion memory 16. Fig. 10 shows a position relation- 
ship between a player P of the game and the game 
device 9 in the two-dimensional area. The X-Y 
coordinates of the coordinate/sound-image-position 
coefficient conversion memory 16 as shown in Fig. 
9 may correspond to the X-Y coordinates of the 
display screen of the display unit 15. In Fig. 9, the 
output channel number CH of a sound source 17 
and some of the coefficients CM1-CM12 which are 
used by the multipliers M1-M12 in the sound- 
image position control apparatus 1 are stored at 
the memory area designated by the X-, Y-coordi- 
nate values which indicates the display position of 
the character C in the display unit 15. For example, 
at an area designated by "AFT, a value "13" is 
stored as the output channel number, while the 
other values "0.6" and "0.8" are also stored as the 
coefficients CM5, CM6 used for the multipliers M5, 
M6 respectively. 

The X/Y coordinates of the coordinate/sound- 
image-position coefficient conversion memory 16 
are set corresponding to those of the actual two- 
dimensional area shown in Fig. 10. In other words, 
the display position of the character C in the dis- 
play unit 15 corresponds to the actual two-dimen- 
sional position of the player as shown in Fig. 10. 
Thus, by adequately setting the parameters, the 
sounds will be produced from the actual position 
corresponding to the display position of the char- 
acter C. Incidentally, the memory area of the 
coordinate/sound-image-position coefficient conver- 
sion memory 16 is set larger than the display area 
of the display unit 15. In this case, the proper 
channel number CH and some of the coefficients 
CM1-CM12 are memorized such that even if the 
character C is located at the coordinates of which 
position cannot be displayed by the display unit 1 5, 
the sounds are produced from the actual position 
corresponding to the coordinates of the character 
C. Moreover, the display position of the character 
C is controlled to be automatically changed in 
response to the progress of the game on the basis 
of the game programs stored in the program mem- 
ory 12, or it is controlled to be changed in re- 
sponse to the manual operation applied to the 



929 A2 14 



controller 10. 

Next, the sound source 17 has plural channels, 
used for the generation of the sounds, which are 
respectively operated in a time-division manner. 
5 Thus, in response to the instruction given from the 
control portion 11, each channel produces a musi- 
cal tone waveform signal. Such musical tone 
waveform signal is delivered to the predetermined 
one or some of eight channels Ch10-Ch17 of the 

w sound-image position control apparatus 1. Particu- 
larly, the musical tone waveform signal regarding to 
the character C is delivered to certain channel Ch 
which is designated by the foregoing output chan- 
nel number CH. As described before, this sound- 

75 image position control apparatus 1 has the elec- 
tronic configuration as shown in Fig. 1(A), wherein 
the predetermined coefficients CM1-CM12 are re- 
spectively applied to the multipliers M1-M12 so as 
to control the sound-image position of each chan- 

20 nel Ch when producing the sounds from the speak- 
ers SP(L), SP(R). 

According to the electronic configurations as 
described heretofore, when the power is applied to 
the game device 9, the control portion 11 is ac- 

25 tivated to execute the programs stored in the pro- 
gram memory 12 so as to progress the game. In 
response to the progress of the game, one of the 
background images BG1, BG2, BG3 is selectively 
read from the visual image information memory 14 

30 so that the selected background image is displayed 
on the display screen of the display unit 15. Simi- 
larly, one of the character images C1, C2, C3 is 
selectively read out so that the selected character 
image is displayed in the display unit 15. Mean- 

35 while, the control portion 11 gives an instruction to 
the sound source 17 so as to produce the musical 
tone waveform signals corresponding to the back- 
ground music in response to the progress of the 
game. In addition, the control portion 11 also in- 

40 structs the sound source 17 to produce the other 
musical tone waveform signals having the musical 
tone characteristics (such as the tone color, tone 
pitch, sound effects, etc.) corresponding to the 
character C. Moreover, the control portion 1 1 reads 

45 out the output channel number CH and coefficient 
CM (i.e., one or some of CM1-CM12) from the 
memory area of the coordinate/sound-image-posi- 
tion coefficient conversion memory 16 correspond- 
ing to the display position of the character C in the 

50 display unit 15, and then the read data are supplied 
to the sound source 17 and sound-image position 
control apparatus 1 respectively. In this case, the 
sound source 17 produces the musical tone 
waveform signal corresponding to the character C, 

55 and this musical tone waveform signal is outputted 
to the sound-image position control apparatus 1 
from the channel Ch which is designated by the 
output channel number CH. The other musical tone 
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waveform signals are also outputted to the sound- 
image position control apparatus 1 from the cor- 
responding channels respectively. In the sound- 
image position control apparatus 1, each of the 
coefficients CM read from the coordinate/sound- 5 
image-position coefficient conversion memory 16 is 
supplied to each of the multipliers M1-M12. Thus, 
the sound-image position of each channel is con- 
trolled to be fixed responsive to the coefficient CM, 
and consequently, the musical sounds are pro- m 
duced from the speakers SP(L), SP(R) at the fixed 
sound-image positions. 

When the player P intentionally operates the 
controller 10 to move the character C, the control 
portion 11 is operated so that the display position 75 
of the character C displayed in the display unit 15 
is moved by the distance corresponding to the 
manual operation applied to the controller 10. In 
this case, new output channel number CH and 
coefficient CM are read from the memory area of 20 
the coordinate/sound-image-position coefficient 
conversion memory 16 corresponding to the new 
display position of the character C, and conse- 
quently, these data are supplied to the sound 
source 17 and sound-image position control ap- 25 
paratus 1 respectively. Thus, the actual sound- 
image position is also moved responsive to the 
movement of the character C. 

According to the present embodiment, when 
the character C representing the visual image of 30 
the air plane is located outside of the display area 
of the display unit 15 and such character C is 
moved closer to the player P from his back, the 
character C is not actually displayed on the display 
screen of the display unit 15. However, since the 35 
foregoing coordinate/sound-image-position coeffi- 
cient conversion memory 16 has the memory area 
which is larger than the display area of the display 
unit 15, the sounds corresponding to the character 
C are actually produced such that the sounds are 40 
coming closer to the player P from his back. As a 
result, the player P can recognize the existence 
and movement of the air plane of which visual 
image is not actually displayed. This can offer a 
brand-new live-audio effect which cannot be ob- 45 
tained from the conventional game device system. 

Incidentally, the present embodiment is de- 
signed to manage the movement of the character C 
in the two-dimensional coordinate system. Of 
course, the present invention is not limited to it, so 50 
that the present embodiment can be modified to 
manage the movement of the character C in the 
three-dimensional coordinate system. In such 
modification, number of the actual speakers are 
increased, and they are arranged in the three- 55 
dimensional space. 

In the present embodiment, the X/Y coordi- 
nates of the display unit 15 are set corresponding 



to those of the actual two-dimensional area. How- 
ever, this embodiment can also modified to simu- 
late the game of the automobile race. In this case, 
only the character C which is displayed in front of 
the player P is displayed in the display unit 15 by 
matching the visual range of the player P with the 
display area of the display unit 15. 

[D] Fourth Embodiment 

Next, the description will be given with respect 
to the fourth embodiment of the present invention, 
wherein the sound-image position control apparatus 
is modified to be applied to the movie system, 
video game device (or television game device) or 
so-called CD-I system in which the sound-image 
position is controlled responsive to the video im- 
age. 

Before describing the fourth embodiment in 
detail in conjunction with Figs. 11 to 13, the de- 
scription will be given with respect to the back- 
ground of the fourth embodiment by referring to 
Figs. 14 and 15. 

First of all, the so-called binaural technique is 
known as the technique which controls and fixes 
the sound-image position in the three-dimensional 
space. According to the known technique, the 
sounds are recorded by use of the microphones 
which are located within the ears of the foregoing 
dummy head, so that the recorded sounds are 
reproduced by use of the headphone set so as to 
recognize the sound-image position which is fixed 
at the predetermined position in the three-dimen- 
sional space. Recently, some attempts are made to 
simulate the tone area which is formed in accor- 
dance with the shape of the dummy head. In other 
words, by simulating the transfer function of the 
sounds which are transmitted in the three-dimen- 
sional space by use of the digital signal processing 
technique, the sound-image position is controlled to 
be fixed in the three-dimensional space. 

The coordinate system of the above-mentioned 
three dimensional space can be defined by use of 
the illustration of Fig. 14. In Fig. 14, V designates 
a distance from the origin "O"; <t> designates an 
azimuth angle with respect to the horizontal direc- 
tion which starts from the origin "0"; 0 designates 
an elevation angle with respect to the horizontal 
area containing the origin "O", thus, the three- 
dimensional space can be defined by the polar 
coordinates in the space. When locating the lis- 
tener or dummy head at the origin 0, its front 
direction can be defined as 4> = 0, whereas its left- 
side direction is defined by 4>>0 and its right-side 
direction is defined by <><0. In addition, the upper 
direction is defined by 0>Q. 

As a model which controls and fixes the sound- 
image position in the three-dimensional space by 
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use of the digital signal processing technique, the 
dummy head is located at the origin O and then 
the impulse signal is produced from the predeter- 
mined point A, for example. Then, the responding 
sounds corresponding to the impulse signal are 5 
sensed by the microphones which are respectively 
located within the ears of the dummy head. These 
sensed sounds are converted into the digital sig- 
nals which are recorded by some recording me- 
dium. These digital signals represent two impulse- w 
response data respectively corresponding to the 
sounds picked up by the left-side and right-side 
ears of the dummy head. These two impulse-re- 
sponse data are converted into the coefficients, by 
which two finite-impulse response digital filters 75 
(hereinafter, simply referred to as FIR filters) are 
respectively given. In this case, the audio signal of 
which sound-image position is not fixed is deliv- 
ered to two FIR filters, through which two digital 
outputs are obtained as the left/right audio signals. 20 
These left/right audio signals are applied to 
left/right inputs of the headphone set, so that the 
listener can hear the stereophonic sounds from this 
headphone set as if those sounds are produced 
from the point A. By changing this point A and 25 
measuring the impulse response, it is possible to 
obtain the other coefficients for the FIR filters. In 
other words, by locating the point A at the desir- 
able position, it is possible to obtain the coeffi- 
cients for the FIR filters, by which the sound-image 30 
position can be fixed at the desirable position. The 
above-mentioned technique offers an effect by 
which the three-dimensional sound-image position 
is determined by use of the sound-reproduction 
system of the headphone set. The same effect can 35 
be embodied by use of the so-called two-speaker 
sound-reproduction system in which two speakers 
are located at the predetermined front positions of 
the listener, which is called a cross-talk canceling 
technique. 40 

According to the cross-talk canceling tech- 
nique, the sounds are reproduced as if they are 
produced from certain position (i.e., position of the 
foregoing virtual speaker) at which the actual 
speaker is not located. Herein, two FIR filters are 45 
required when locating one virtual speaker, 
hereinafter, a set of two FIR filters will be called as 
a sound-directional device. 

Fig. 15 is a block diagram showing an example 
of the virtual-speaker circuitry which employs the 50 
above-mentioned sound-directional device. In Fig. 
15, 102-104 designate sound-directional devices, 
each of which contains two FIR filters. This drawing 
only illustrates three sound-directional devices 102- 
104, however, there are actually provided several 55 
hundreds of the sound-directional devices. Thus, it 
is possible to locate hundreds of virtual speakers in 
a close-tight manner with respect to all of the 



directions of the polar-coordinate system. These 
virtual speakers are not merely arranged along with 
a spherical surface with respect to the same dis- 
tance r, but they are also arranged in a perspective 
manner with respect to different distances r. A 
selector 101 selectively delivers the input signal to 
one of the sound-directional devices such that the 
sounds will be produced from the predetermined 
one of the virtual speakers, thus controlling and 
fixing the sound-image position in the three-dimen- 
sional space. Incidentally, adders 105, 106 output 
their addition results as the left/right audio outputs 
respectively. 

The above-mentioned example can be modi- 
fied such that one sound-directional device is not 
fixed corresponding to one direction of producing 
the sound. In other words, by changing the coeffi- 
cients of the FIR filters contained in one sound- 
directional device, it is possible to move the sound- 
image position by use of only one sound-direc- 
tional device. 

In the meantime, some movie theater employs 
so-called surround acoustic technique which uses 
four or more speakers. Therefore, the sounds are 
produced from one or some speakers in response 
to the video image. 

When embodying such surround acoustic tech- 
nique by use of the former virtual-speaker system 
providing hundreds of sound-directional devices, it 
is necessary to provide hundreds of FIR filters, 
which enlarges the scale of the system so that the 
cost of the system will be eventually raised up. 
Even in the case of the latter system which pro- 
vides only one sound-directional device, it is nec- 
essary to provide hundreds of coefficients used for 
the FIR filter, which is not realistic. Because, it is 
very difficult to control or change so many number 
of coefficients in a real-time manner. Further, when 
embodying the foregoing surround acoustic tech- 
nique in the movie theater, it is necessary to pro- 
vide a plenty of amplifiers and speakers, which 
eventually raises the cost of the facilities. 

(a) Configuration of Fourth Embodiment 

Next, the detailed description will be given with 
respect to the fourth embodiment of the present 
invention. Fig. 11 is a block diagram showing the 
whole configuration of the video game system. 
Herein, a game device 21 is designed to produce a 
video signal VS, a left-side musical tone signal ML, 
a right-side musical tone signal MR, a sound effect 
signal EFS, a panning signal PS and a scene- 
identification signal SCS. When receiving the 
sound effect signal EFS, panning signal PS and 
scene-identification signal SCS, a sound-image po- 
sition control apparatus 22 imparts the fixed sound 
image to the sound effect signal EFS, thus produc- 
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ing two signals EFSL, EFSR. Then, an adder 25 
adds the signals EFSR and MR together, while an 
adder 26 adds the signals EFSL and ML together. 
The results of the additions respectively performed 
by the adders 25, 26 are supplied to an amplifier 
24. The amplifier 24 amplifies these signals so as 
to respectively output the amplified signals to 
left/right loud-speakers (represented by 43, 44 in 
Fig. 13). In the meantime, the video signal VS is 
supplied to a video device 23, so that the video 
image is displayed for the person. 

The game device 21 is configured as the 
known video game device which is designed such 
that responsive to the manipulation of the player of 
the game, the scene displayed responsive to the 
video signal VS is changed or the position of the 
character image is moved. During the game, the 
musical tone signals ML, MR are outputted so as to 
playback the background music. In addition, to this 
background music, the other sounds are also pro- 
duced. For example, the sounds corresponding to 
the character image which is moved responsive to 
the manipulation of the player, or the other sounds 
corresponding to the other character images which 
are automatically moved under control of the con- 
trol unit built in the game device 21 are produced 
by the sound effect signal EFS. In case of the 
game of the automobile race, the engine sounds of 
the automobiles are automatically produced. 

The scene-identification signal SCS is used for 
determining the position of the virtual speaker in 
accordance with the scene. Every time the scene is 
changed, this scene-identification signal SCS is 
produced as the information representing the 
changed scene. Such scene-identification signal 
SCS is stored in advance within a memory unit (not 
shown) which is built in the game device 21 . More 
specifically, this signal is stored at the predeter- 
mined area adjacent to the area storing the data 
representing the background image with respect to 
each scene of the game. Thus, when the scene is 
changed, this signal is simultaneously read out. 

On the other hand, the panning signal PS re- 
presents certain position which is located between 
two virtual speakers. By varying the value of this 
panning signal PS between "0" and "1", it is 
possible to freely change the sound-image position 
corresponding to the sound produced responsive to 
the sound effect signal EFS between two virtual 
speakers. In the present embodiment, the pro- 
grams of the game contain the operation routine for 
the panning signal PS, by which the panning signal 
PS is computed on the basis of the scene-iden- 
tification signal SCS and the displayed position of 
the character image corresponding to the sound 
effect signal EFS. Of course, such computation of 
the panning signal PS can be omitted, so that in 
response to the position of the character, the game 



device 21 automatically reads out the panning sig- 
nal PS which is stored in advance in the memory 
unit. Incidentally, the present embodiment is de- 
signed such that two virtual speakers are emerged, 

5 which will be described later in detail. 

Fig. 12 is a block diagram showing an internal 
configuration of the sound-image position control 
apparatus 22. Herein, a control portion 31 is config- 
ured as the central processing unit (i.e., CPU), 

70 which performs the overall control on this appara- 
tus 22. This control portion 31 receives the fore- 
going scene-identification signal SCS and panning 
signal PS. A coefficient memory 32 stores the 
coefficients of the FIR filters. As described before, 

75 the impulse response is measured with respect to 
the virtual speaker which is located at the desirable 
position, so that the above-mentioned coefficients 
are determined on the basis of the result of the 
measurement. In order to locate the virtual speaker 

20 at the optimum position corresponding to the scene 
of the game, the coefficients for the FIR filters are 
computed in advance with respect to several posi- 
tions of the virtual speaker, and consequently, 
these coefficients are stored at the addresses of 

25 the memory unit corresponding to the scene-iden- 
tification signal SCS. As described before, each of 
sound-directional devices 33, 34 is configured by 
two FIR filters. The coefficient applied to the FIR 
filter can be changed by the coefficient data given 

30 from the control portion 31. 

In response to the scene-identification signal 
SCS, the control portion 11 reads out the coeffi- 
cient data, respectively corresponding to the virtual 
speakers L, R, from the coefficient memory 32, and 

35 consequently, the read coefficient data are respec- 
tively supplied to the sound-directional devices 33, 
34. When receiving the coefficients, each of the 
sound-directional devices 33, 34 performs the pre- 
determined signal processing on the input signal of 

40 the FIR filters, thus locating the virtual speaker at 
the optimum position corresponding to the scene- 
identification signal SCS. 

The sound effect signal EFS is allocated to the 
sound-directional devices 33, 34 via multipliers 35, 

45 36 respectively. These multipliers 35, 36 also re- 
ceive the multiplication coefficients respectively 
corresponding to the values "PS", "1-PS" from the 
control portion. Herein, the value "PS" represents 
the value of the panning signal PS, while the value 

so "1-PS" represents the one's complement of the 
panning signal PS. The outputs of first FIR filters in 
the sound-directional devices 33, 34 are added 
together by an adder 37, while the other outputs of 
second FIR filters in the sound-directional devices 

55 33, 34 are added together by another adder 38. 
Therefore, these adders 37, 38 output their addition 
results as signals for the speakers 43, 44 respec- 
tively. These signals are supplied to a cross-talk 
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canceler 39. 

The cross-talk canceler 39 is provided to can- 
cel the cross-talk component included in the 
sounds. For example, the cross-talk phenomenon 
must be occurred when producing the sounds from 
the speakers 43, 44 in Fig. 13. Due to this cross- 
talk phenomenon, the sound component produced 
from the left-side speaker affects the sound which- 
is produced from the right-side speaker for the 
right ear of the listener, while the sound component 
produced from the right-side speaker affects the 
sound which is produced from the left-side speaker 
for the left ear of the listener. Thus, in order to 
cancel the above-mentioned cross-talk compo- 
nents, the cross-talk canceler 39 performs the con- 
volution process by use of the phase-inverted sig- 
nal having the phase which is inverse to that of the 
cross-talk component. Under operation of this 
cross-talk canceler 39, the outputs of the sound- 
directional device 33 are converted into the sounds 
which are roughly heard by the left ear only from 
the left-side speaker, while the outputs of the 
sound-directional device 34 are converted into the 
sounds which are roughly heard by the right ear 
only from the right-side speaker. Such sound al- 
location can roughly embody the situation in which 
the listener hears the sounds by use of the head- 
phone set. 

Meanwhile, the cross-talk canceler 39 receives 
a cross-talk bypass signal BP from the control 
portion 31. This cross-talk bypass signal BP is 
automatically produced by the control portion 31 
when inserting the headphone plug into the head- 
phone jack (not shown). When the headphone plug 
is not inserted, the cross-talk bypass signal BP is 
turned off, so that the sounds are reproduced from 
two speakers while canceling the cross-talk compo- 
nents as described before. On the other hand, 
when the headphone plug is inserted, the cross-talk 
canceling operation is omitted, so that the signals 
are supplied to the headphone set from which the 
sounds are reproduced. 

Next, the description will be given with respect 
to the method how to control and fix the sound- 
image position by the panning signal PS. When the 
value of the panning signal PS is equal to zero, the 
foregoing sound effect signal EFS is supplied to 
the sound-directional device 34 only. Thus, the 
sound-image position is fixed at the position of the 
virtual speaker (i.e., position of the speaker 45 in 
Fig. 13) which is located by the sound-directional 
device 34. On the other hand, when the value of 
the panning signal PS is at "1", the sound effect 
signal EFS is supplied to the sound-directional 
device 33 only, and consequently, the sound-image 
position is fixed at the position of the virtual speak- 
er (i.e., position of a speaker 46) which is located 
by the sound-directional device 33. When the value 



of the panning signal PS is set at a point between 
"0" and "1", the sound-image position is fixed at 
an interior-division point corresponding to the pan- 
ning signal PS between the virtual speakers 45, 46. 

5 

(b) Operation of Fourth Embodiment 

Next, description will be given with respect to 
the operation of the fourth embodiment by referring 

70 to Fig. 13. In Fig. 13, a player 41 is positioned at 
the center, whereas the left-side speaker 43 is 
located at the left/front-side position from the play- 
er 41 which is defined by <f> = 45' , 0=0° , 
r=1.5m, while the right-side speaker 44 is located 

15 at the right/front-side position from the player 41 
which is defined by </> = -45* ,6=0* , r=1.5m. In 
front of the player 41, there is located a display 
screen 42 of the video device 23. In the present 
embodiment, this display screen 42 has a flat- 

20 plate-like shape, however, it is possible to form this 
screen by the curved surface which surrounds the 
player 41 . 

For example, the player 41 plays the game and 
the duel scene of the Western is displayed. In this 

25 case, the game device 21 outputs the scene-iden- 
tification signal SCS to the control portion 31 in the 
sound-image position control apparatus 22, wherein 
this scene-identification signal SCS has the pre- 
determined scene-identifying value, e.g., four-bit 

30 data "0111". Then, the control portion 31 reads out 
coefficient data CL, corresponding to the scene- 
identification signal SCS, from the coefficient mem- 
ory 32, wherein this coefficient data CL represents 
the coefficient for the FIR filter which corresponds 

35 to the position of the left-side virtual speaker 45 
(defined by <t>=85* ,6=0* , r = 3.5m). This coeffi- 
cient data CL is supplied to the sound-directional 
device 33. In addition, the control portion 31 also 
read out another coefficient data CR representing 

40 the coefficient for the FIR filter which corresponds 
to the position of the right/upper-side virtual speak- 
er 46 (defined by <fr = -40* . 0=65° , r = 15.0m). 
This coefficient data CR is supplied to the sound- 
directional device 34. Thus, the virtual speakers 45, 

45 46 are located at their respective positions as 
shown in Fig. 13. 

The game device 21 produces the musical 
tone signals ML, MR which are sent to the speak- 
ers 43, 44 via the adders 25, 26 and amplifier 24, 

50 whereas the music which is suitable for the duel 
scene is reproduced, while the other background 
sounds such as the wind sounds are also repro- 
duced, regardless of the sound-image position con- 
trol. In response to a shot action of a gunfighter 

55 which is the displayed image and plays an enemy 
role for the player 41 in the gunfight game, the 
sound effect signal EFS representing a gunshot 
sound is supplied to the sound-image position con- 
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trol apparatus 22. In this case, if the value of the 
panning signal PS is equal to zero, the gunshot is 
merely sounded from the position of the virtual 
speaker R46. Such sound effect corresponds to the 
scene in which the gunfighter shoots a gun by 
aiming at the player 41 from the second floor of the 
saloon. On the other hand, if the value of the 
panning signal PS is equal to "1", the gunshot may 
be sounded in the scene in which the gunfighter is 
placed at the left-side position very close to the 
player 41 and then the gunfighter shoots a gun at 
the player 41 . If the value of the panning signal PS 
is set at certain value between "0" and "1", the 
gunfighter is placed at certain interior-division point 
on the line connected between the virtual speakers 
45, 46, and then the gunshot is sounded. 

The game device 21 is designed such that 
even in the same duel scene of the Western, every 
time the position of the enemy is changed, new 
scene-identification signal SCS (having a new bi- 
nary value such as "1010") is produced and out- 
putted to the sound-image position control appara- 
tus 22. In other words, the change of the position of 
the enemy is dealt as the change of the scene. 
Thus, the virtual speakers will be located again in 
response to the new scene. 

Other than the above-mentioned Western 
game, the game device 21 can also play the auto- 
mobile race game. Herein, the game device 21 
outputs a new scene-identification signal SCS 
(having a binary value such as "0010"), by which 
the control portion 31 reads out two coefficient data 
respectively corresponding to the right/front-side 
virtual speaker and right/back-side virtual speaker. 
These coefficient data are respectively supplied to 
the sound-directional devices 33, 34. In this case, 
the foregoing signals ML, MR represent the back- 
ground music and the engine sounds of the auto- 
mobile to be driven by the player 41. Further, the 
foregoing signal EFS represents the engine sounds 
of the other automobiles which will be running in 
the race field as the displayed images. On the 
basis of the foregoing operation routine, the pan- 
ning signal PS is computed and renewed in re- 
sponse to the position relationship between the 
player's automobile and the other automobiles. If 
another automobile is running faster than the play- 
er's automobile so that another automobile will get 
ahead of the player's automobile, the value of the 
panning signal PS is controlled to be gradually 
increased from "0" to "1". Thus, in response to the 
scene in which another automobile gets ahead of 
the player's automobile, the sound-image position 
of the engine sound of another automobile is con- 
trolled to be gradually moved ahead. 

As described above, the fourth embodiment is 
applied to the game device. However, it is possible 
to modify the present embodiment such that the 
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sound-image position control is performed in re- 
sponse to the video scene played by the video disk 
player. Or, it is possible to apply the present em- 
bodiment to the CD-I system. In this case, the 

5 foregoing scene-identification signal SCS and pan- 
ning signal PS can be recorded at the sub-code 
track provided for the audio signal. 

Further, the present embodiment uses two 
sound-directional devices, however, it is possible to 

10 modify the present embodiment such that three or 
four sound-directional devices are provided to cope 
with more complicated video scenes. In this case, 
the complicated control must be performed on the 
panning signal PS. However, it is not necessary to 

15 provide hundreds of sound-directional devices, or it 
is not necessary to simultaneously change hun- 
dreds of coefficients for the FIR filters. 

Moreover, the sound-directional device of the 
present embodiment is configured by the FIR fil- 

20 ters, however, this device can be configured by the 
infinite-impulse response digital filters (i.e., MR fil- 
ters). For example, the so-called notch filter is 
useful when fixing the sound-image position with 
respect to the elevation-angle direction. Further, it 

25 is also known that the band-pass filter controlling 
the specific frequency-band is useful when control- 
ling the sound-image position with respect to the 
front/back direction. When embodying such filter 
by use of the MR filters, the fixing degree of the 

30 sound-image position may be reduced as com- 
pared to the FIR filters. However, the MR filter has a 
simple configuration as compared to the FIR filter, 
so that the number of the coefficients can be 
reduced. In short, the MR filter is advantageous in 

35 that the controlling can be made easily. 

Lastly, this invention may be practiced or em- 
bodied in still other ways without departing from 
the spirit or essential character thereof as de- 
scribed heretofore. Therefore, the preferred em- 

40 bodiments described herein are illustrative and not 
restrictive, the scope of the invention being in- 
dicated by the appended claims and all variations 
which come within the meaning of the claims are 
intended to be embraced therein. 

45 

Claims 

1. A sound-image position control apparatus char- 
acterized by comprising: 
50 a signal mixing means (MTR1) for mixing 

plural audio signals supplied thereto in accor- 
dance with a predetermined signal mixing pro- 
cedure so as to output plural mixed signals; 
and 

55 a virtual-speaker position control means 

(DL10-DL13, KL10-KL13, KR10-KR13, AD10- 
AD13) for applying different delay times to 
each of said plural mixed signals so as to 
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output delayed signals as right-side and left- 
side audio signals to be supplied to right-side 
and left-side speakers (SP(R), SP(L)), thus 
controlling sound-image positions formed by 
virtual speakers (VS10-VS13) to be emerged 
as sound-producing points each of which virtu- 
ally produces sounds corresponding to each of 
said plural mixed signals, 

whereby sounds applied with a stereo- 
phonic effect and a clear sound-image dis- 
crimination effect are to be actually produced 
from said right-side and left-side speakers as if 
the sounds are virtually produced from said 
virtual speakers of which positions are deter- 
mined under control of said virtual-speaker po- 
sition control means. 

2. A sound-image position control apparatus char- 
acterized by comprising: 

a first mixing means (MTR1) for mixing 
plural audio signals supplied thereto in accor- 
dance with a predetermined signal mixing pro- 
cedure so as to output plural mixed signals; 

a plurality of delay means (DL10-DL13) 
each having two delay times, each of said 
delay means delaying one of said plural mixed 
signals by said two delay times respectively so 
as to output two delayed signals as right-side 
and left-side delayed signals respectively used 
for right-side and left-side speakers (SP(R), 
SP(L)); and 

a second mixing means (KL10-KL13, 
KR10-KR13, AD10-AD13) for mixing said right- 
side delayed signals respectively outputted 
from said plurality of delay means together so 
as to output a right-side audio signal, said 
second mixing means also mixing said left- 
side delayed signals together so as to output a 
left-side audio signal, so that said right-side 
and left-side speakers produce sounds, applied 
with a stereophonic effect, on the basis of said 
right-side and left-side audio signals, 

whereby a plurality of virtual speakers 
(VS10-VS13) are emerged as sound-producing 
points of which positions are controlled by said 
plurality of delay means. 

3. A sound-image position control apparatus char- 
acterized by comprising: 

a sound source means (17) for generating 
plural audio signals; 

a virtual-speaker position control means 
(DL10-DL13, KL10-KL13, KR10-KR13, AD10- 
AD13) for receiving said plural audio signals 
and then applying different delay times to each 
of said plural audio signals so as to output 
delayed signals as right-side and left-side 
audio signals to be supplied to right-side and 
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left-side speakers (SP(R), SP(L)), thus control- 
ling sound-image positions formed by virtual 
speakers (VS10-VS13) to be emerged as 
sound-producing points each of which virtually 
produces sounds corresponding to each of 
said plural audio signals; and 

a display means (15) for displaying a pre- 
determined animated image on a display 
screen thereof, said animated image corre- 
sponding to the sounds to be virtually pro- 
duced from each of said virtual speakers, 
wherein a display position of said animated 
image corresponds to a position of the sound- 
producing point embodied by said virtual 
speakers so that the position of the sound- 
producing point corresponding to said animat- 
ed image is moved in accordance with a 
movement of said animated image on the dis- 
play screen of said display means. 



4. A sound-image position control apparatus as 
defined in claim 1 wherein said signal mixing 
means is a matrix controller (MTR1 ) containing 
plural multipliers (M1-M12) and plural adders 

25 (IN10-IN13) of which connection pattern is 

changed over in accordance with a change of 
a signal mixing procedure. 

5. A sound-image position control apparatus as 
30 defined in claim 1 wherein said virtual-speaker 

position control means further comprises: 

a plurality of delay circuits (DL10-DL13) 
each having two delay times, wherein each of 
said plurality of delay circuits delays one of 
35 said plural mixed signals by two delay times 

respectively so as to output two delayed sig- 
nals; and 

an allocation ratio applying means (KL10- 
KL13. KR10-KR13, AD10-AD13) for applying a 
40 predetermined allocation ratio to said delayed 

signals, thus allocating them as said right-side 
and left-side audio signals to be respectively 
supplied to said right-side and left-side speak- 
ers. 

45 

6. A sound-image position control system char- 
acterized by comprising: 

a means (21) for producing a video signal 
and an audio signal which are related to each 
so other; 

a scene-identification signal producing 
means (21) for producing a scene-identification 
signal (SCS) corresponding to each scene of a 
display image; 
55 a plurality of speakers (43, 44); 

a sound-image forming means (33, 34) for 
driving said speakers by performing a pre- 
determined signal processing on said audio 
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signal so as to form a sound image at a 
desirable position which is not only located 
within linear space connected between said 
speakers; and 

a control means (31) for changing over the 5 
contents of the signal processing in response 
to said scene-identification signal so as to con- 
trol and fix a sound-image position of said 
audio signal. 

;o 

7. A sound-image position control system char- 
acterized by comprising: 

an audio/video information producing 
means (21) for producing a video signal and an 
audio signal which are related to each other, 75 
said means also producing a scene-identifica- 
tion signal (SCS) corresponding to each scene 
of a display image which is displayed by a 
display unit (23, 42); 

at least two speakers (43, 44) which are 20 
respectively located at predetermined posi- 
tions; 

a sound-image forming means (33, 34) for 
performing a predetermined signal processing 
on said audio signal so that said apparatus 25 
forms a sound image at a desirable position in 
a three-dimensional space surrounding a per- 
son who watches the display image; and 

a control means (31) for changing over the 
contents of the signal processing in response 30 
to said scene-identification signal so as to con- 
trol a sound-image position of said audio sig- 
nal. 



11. A sound-image position control apparatus char- 
acterized by comprising: 

at least two virtual-speaker means (DL10- 
DL13, KL10-KL13, KR10-KR13) each of which 
outputs plural signals; 

an allocation means (MTR1) which re- 
ceives an input signal thereof so as to allocate 
it to said virtual-speaker means; 

addition means (AD10-AD13) for adding 
plural output signals of said virtual-speaker 
means; and 

a plurality of real-speaker means (SP(L), 
SP(R)) for producing sounds corresponding to 
addition results of said addition means. 

12. A sound-image position control apparatus as 
defined in claim 11 further providing a cross- 
talk canceling means (XTC) between said addi- 
tion means and said plurality of real-speaker 
means, said cross-talk canceling means can- 
celing a cross-talk phenomenon which is oc- 
curred between sounds produced from said 
plurality of real-speaker means. 



8. A sound-image position control system as de- 35 
fined in claim 7 wherein said sound-image 
forming means includes at least two virtual- 
speaker position control means (33, 34) which 
respectively perform predetermined signal pro- 
cessings corresponding to said scene-identifi- aq 
cation signal on said audio signal so as to form 

at least two virtual speakers by which the 
sound image corresponding to said audio sig- 
nal is formed. 

45 

9. A sound-image position control system as de- 
fined in claim 8 wherein said audio/video in- 
formation producing means also produces a 
panning signal (PS) by which the sound-image 
position is located at certain interior-division 50 
point in linear space connected between said 
virtual speakers. 



10. A sound-image position control system as de- 
fined in claim 7 wherein said sound-image 55 
forming means is configured by use of a finite- 
impulse response digital filter (i.e., FIR filter). 
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© Sound-image position control apparatus. 



® In order to obtain a sound-broadened image and 
a clear sound-image discrimination image when pro- 
ducing plural kinds of sounds, the electronic musical 
instrument and the like provides a sound-image po- 
sition control apparatus. This apparatus at least pro- 
^ vides a signal mixing portion (e.g., matrix controller; 
^ MTR1) and a virtual-speaker position control portion 
©2> (DL10-DL13, KL10-KL13, KR10-KR13, AD10-AD13). 

Herein, the signal mixing portion mixes plural audio 
$@ signals supplied from a sound source (17) and the 
*A like in accordance with a predetermined signal mix- 
® ing procedure so as to output plural mixed signals. 
^ In order to control positions of virtual speakers 
yy (VS10-VS13) which are emerged as sound-produc- 
ing points as if each kind of sounds is produced 
from each of these points, the virtual-speaker posi- 



tion control portion applies different delay times to 
each of plural mixed signals so as to output delayed 
signals as right-side and left-side audio signals to be 
respectively supplied to right-side and left-side 
speakers (SP(R), SP(L)). Thus, the sound-image po- 
sitions formed by the virtual speakers are controlled 
well, so that the person can clearly discriminate and 
recognize each of the sound-image positions. When 
applying this apparatus to the game device providing 
a display unit which displays an animated image 
representing a visual image of the air plane and the 
like, by adequately controlling the sound-image posi- 
tion, it is possible to obtain a brand-new live-audio 
effect, by which the point of producing the sounds 
corresponding to the animated image can be moved 
in accordance with the movement of the animated 
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Description 

The present invention relates to a sound-image po- 
sition control apparatus which is suitable for use in the 
electronic musical instruments, audio-visual devices 5 
and the like so as to eventually perform the sound-image 
localization. 

As the device which offers the person the sound- 
broadened image, there are provided the stereo-chorus 
device, reverberation device and the like. Herein, the 
former one is designed to produce the sound of which 
phase is slightly shifted as compared to that of the orig- 
inal sound so that this phase-shifted sound and the orig- 
inal sound are alternatively produced from the left and 
right loud-speakers, while the latter one is designed to 
impart the reverberation effect to the sounds. 

In addition, there is another device, called the pan- 
ning device. This panning device is designed to provide 
the predetermined output-level difference between the 
sounds which are respectively produced from the left 
and right loud-speakers, resulting that the stereophonic 
effect or stereo-impressive image is applied to the 
sounds. 

The above-mentioned stereo-chorus device or re- 
verberation device can enlarge the sound-broadened 
image. However, there is a drawback in that the sound- 
distribution image which is sensed by the listener must 
become unclear when enlarging the sound-broadened 
image. Herein, the sound-distribution image is defined 
as a degree of discrimination in which the person who 
listens to the music from the audio device can specifi- 
cally discriminate the sound of certain instrument from 
the other sounds. For example, when listening to the 
music played by the guitar and keyboard by the audio 
device having a relatively good sound-distribution im- 
age control, the person can discriminate the respective 
sounds as if the guitar sound is produced from the pre- 
determined left-side side position, while the keyboard 
sound is produced from the predetermined right-side 
position (hereinafter, such virtual position will be re- 
ferred to as the sound-image position). When listening 
to the music by use of the aforementioned stereo-chorus 
device or reverberation device, it is difficult for the per- 
son to clearly discriminate the sound-image positions. 

In the panning device, the sound-image position 
must be fixed at the predetermined position disposed 
on the line connecting the left and right loud-speakers 
on the basis of the sound-image localization technique, 
resulting that the sound-broadened image cannot be 
substantially obtained. In other words, when simultane- 
ously producing plural sound each having a different 
sound-image position, the panning device merely func- 
tions to roughly mix up those sound so that the clear 
sound-image positions cannot be obtained. 

In the meantime, the panning device is frequently 
equipped with or built in the electronic musical instru- 
ment when simulating the sounds of the relatively large- 
scale instruments such as the piano, organ and vibra- 



phone. In such instrument (e.g., piano), the sound-pro- 
ducing positions must be moved accompanied with the 
progression of notes, thus, the panning device functions 
to simulate such movement of the sound-producing po- 
sitions. 

However, the panning device also suffers from the 
aforementioned drawback. More specifically, the pan- 
ning device can offer certain degree of panning effect 
when simulating the sounds, however, it is not possible 
to clearly discriminate the sound-image position of each 
of the sounds to be produced. In short, the panning de- 
vice cannot perform the accurate simulation with re- 
spect tothe discrimination of the sound-image positions. 

EP-A-0 357 402 discloses a sound imaging method 
and apparatus. The sound processing according to this 
reference involves dividing each monaural or single 
channel signal into two signals and then adjusting the 
differential phase and amplitude of the two channel sig- 
nals on a frequency dependent basis in accordance with 
an empirically derived transfer function that has a spe- 
cific phase and amplitude adjustment for each predeter- 
mined frequency interval over the audio spectrum. For 
each different sound source location a specific transfer 
function must be empirically derived and stored. If a 
sound source should be made to appear to move, a 
large number of different transfer functions must be pro- 
vided and stored in the apparatus. Since each transfer 
function is empirically derived this method is very time 
consuming. 

US-A-4,685,134 discloses a multi-channel compu- 
ter generated sound synthesis system. The system in- 
cludes in each channel a sound generator having a pro- 
grammable delay and an adjustable gain. This known 
system is expensive since each of the channel requires 
at least one separate sound generator. If different kind 
of sounds should be produced simultaneously the 
number of the sound generators and thus the costs of 
the sound system would increase drastically. 

It is accordingly a primary object of the present in- 
vention to provide a sound-image position control appa- 
ratus by which even when simultaneously producing 
plural sounds each having a different sound-image po- 
sition, it is possible to clearly discriminate the sound- 
image position of each of the sounds to be produced. 

It is another object of the present invention to pro- 
vide a sound-image position control apparatus which 
can offer the sound-broadened effect, stereophonic ef- 
fect or stereo-impressive image when simultaneously 
producing plural sounds each having a different sound- 
image position. 

It is a further object of the present invention to pro- 
vide a sound-image position control apparatus which 
can offer a sound-image localization with a simple con- 
figuration of the apparatus. 

These objects are achieved according tothe inven- 
tion by a sound image position control apparatus as de- 
fined by the appended independent claim 1 , with advan- 
tageous embodiments defined by the dependent claims. 
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In the drawings: 

Fig. 1(A) is a block diagram showing an electronic 
configuration of a sound-image position control ap- 
5 paratus according to a first embodiment of the 
present invention; 

Fig. 1(B) is a plan view illustrating a position rela- 
tionship between the performer and speakers; 
Fig. 2(A) is a block diagram showing another exam- 
10 pie of the arrangement of circuit elements in a ma- 
trix controller; 

Fig. 2(B) is a plan view illustrating another example 
of the position relationship between the performer 
and speakers; 

15 Fig. 3(A) is a block diagram showing a detailed elec- 
tronic configuration of a cross-talk canceler shown 
in Fig. 1(A); 

Fig. 3(B) is a plan view illustrating another example 
of the position relationship between the performer 

20 and speakers; 

Fig. 4 is a plan view illustrating a fundamental po- 
sition relationship between the performer and 
speakers according to the present invention; 
Fig. 5 is a block diagram showing a modified exam- 

25 pie of the first embodiment; 

Fig. 6 is a block diagram showing an electronic con- 
figuration of a sound-image position control appa- 
ratus according to a second embodiment of the 
present invention; 

30 Fig. 7 is a drawing showing a relationship between 
the person and virtual sound source; 
Fig. 8 is a block diagram showing an electronic con- 
figuration of a game device to which a sound-image 
position control apparatus according to a third em- 

35 bodiment of the present invention is applied; 

Fig. 9 is a drawing showing a two-dimensional 
memory map of a coordinate/sound-image-position 
coefficient conversion memory shown in Fig. 8; 
Fig. 10 is a plan view illustrating a position relation- 

40 ship between the player and game device; 

Fig. 11 is a block diagram showing an electronic 
configuration of a video game system; 
Fig. 12 is a block diagram showing an electronic 
configuration of a sound-image position control ap- 

45 paratus, shown in Fig. 1 1 , according to a fourth em- 
bodiment of the present invention; 
Fig. 13 is a drawing illustrating a position relation- 
ship among a listener, loud-speakers and a video 
screen; 

50 Fig. 14 illustrates a polar-coordinate system which 
is used for defining a three-dimensional space; and 
Fig. 1 5 is a block diagram showing a typical exam- 
ple of a virtual-speaker system, of which concept is 
applied to the fourth embodiment. 

55 



According to the fundamental configuration of the 
present invention, the sound-image position control ap- 
paratus comprises a signal mixing portion and a virtual- 
speaker position control portion. Herein, the signal mix- 
ing portion mixes plural audio signals supplied thereto 
in accordance with a predetermined signal mixing pro- 
cedure so as to output plural mixed signals. The virtual- 
speaker position control portion applies different delay 
times to each of plural mixed signals so as to output de- 
layed signals as right-side and left-side audio signals to 
be respectively supplied to right-side and left-side 
speakers. In this case, some virtual speakers are virtu- 
ally emerged as sound-producing points as if each of 
the sounds is produced from each of these points. Thus, 
sound-image positions formed by the virtual speakers 
are controlled in accordance with plural mixed signals. 

Under effect of the aforementioned configuration of 
the present invention, the sounds applied with the ster- 
eophonic effect and clear sound-image discrimination 
effect are to be actually produced from the right-side and 
left-side speakers as if the sounds are virtually produced 
from the virtual speakers of which positions are deter- 
mined under control of the virtual-speaker position con- 
trol portion. 

When applying this apparatus to the game device 
providing a display unit which displays an animated im- 
age representing an image of the air plane and the like, 
by adequately controlling the sound-image position, it is 
possible to obtain a brand-new live-audio effect, by 
which the point of producing the sounds corresponding 
to this animated image is moved in accordance with the 
movement of the animated image which is moved by the 
player of the game. 

Moreover, the present invention can be easily mod- 
ified to be applied to the movie system or video game 
device in which the sound-image position is controlled 
responsive to the video image. This system comprises 
an audio/video signal producing portion; a scene-iden- 
tification signal producing portion; a plurality of speak- 
ers; a sound-image forming portion; and a control por- 
tion. 

The above-mentioned scene-identification signal 
producing portion outputs a scene-identification signal 
in response to a scene represented by the video signal. 
The sound-image forming portion performs the prede- 
termined processings on the audio signals so as to drive 
the speakers. Under effect of such signal processings, 
the speakers produce the sounds of which sound-image 
positions are fixed at the desirable positions departing 
from the linear spaces directly connecting the speakers. 
The control portion controls the contents of the signal 
processings so as to change over the fixed sound-image 
position in response to the scene-identification signal. 

Further objects and advantages of the present in- 
vention will be apparent from the following description, 
reference being had to the accompanying drawings 
wherein the preferred embodiments of the present in- 
vention are clearly shown. 



Now, description will be given with respect to the 
embodiments of the present invention by referring to the 
drawings, wherein the predetermined position relation- 
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ship is fixed between a performer P and an instrument 
I as shown in Fig. 4. In the description, the lateral direc- 
tion indicates an arrow direction "a", while the longitudi- 
nal direction indicates an arrow direction "b" as shown 
in Fig. 4. 

[A] First Embodiment 

(1) Configuration 

Fig. 1 (B) is a plan view illustrating a position rela- 
tionship between a person M (i.e., performer) and an 
electronic musical instrument containing two speakers 
(i.e., loud-speakers). Herein, KB designates a keyboard 
providing plural keys, wherein when depressing a key, 
a tone generator (not shown) produces a musical tone 
waveform signal having the pitch corresponding to the 
depressed key. SP(L) and SP(R) designate left and right 
speakers respectively. These speakers SP(L), SP(R) 
are respectively arranged at the predetermined left-side 
and right-side positions of the upper portion of the in- 
strument. 

Fig. 1 (A) is a block diagram showing an electronic 
configuration of a sound-image position control appara- 
tus 1 according to a first embodiment of the present in- 
vention. This apparatus 1 provides eight channels re- 
spectively denoted by numerals Ch10 to Ch17 (given 
with a general numeral n Ch"), wherein each channel Ch 
receives the musical tone waveform signal produced 
from the tone generator. Specifically, the musical tone 
waveform signal supplied to each channel Ch has the 
allocated frequency domain corresponding to some mu- 
sical notes (hereinafter, referred to as the allocated tone 
area). 

More specifically, the allocation of the tone areas is 
given as follows: the musical tone waveform signal of 
which tone area corresponds to the lowest-pitch note to 
C1 note is supplied to the channel Ch10, while the mu- 
sical tone waveform signal of which tone area corre- 
sponds to C#1 note to C2 note is supplied to the channel 
Ch11. Similarly, the tone area of C#2 to F2 is allocated 
to the channel Ch12; the tone area of F#2 to C3 is allo- 
cated to the channel Ch13; the tone area of C#3 to F3 
is allocated to the channel Chf 4; the tone area of F#3 
to C4 is allocated to the channel Ch15; the tone area of 
C#4 to C#5 is allocated to the channel Ch16; and the 
tone area corresponding to the D5 note to the highest- 
pitch note is allocated to the channel Ch17. 

Next, M1 to M12 designate multipliers which multi- 
ply the musical tone waveform signal supplied thereto 
by respective coefficients CM1 to CM12. IN10 to IN13 
designate adders, each of which receives the outputs of 
some multipliers. The above-mentioned elements, i.e., 
multipliers M1 to M12, adders IN10 to IN13 and chan- 
nels Ch1 0 to Ch1 7 are assembled together into a matrix 
controller MTR1. Herein, the connection relationship 
and arrangement relationship among those elements of 
the matrix controller MTR1 can be arbitrarily changed in 
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response to the control signal and the like. Incidentally, 
the detailed explanation of this matrix controller MTR1 
will be given later. 

Meanwhile, DL10 to DL1 3 designate delay circuits 
s which respectively delays the outputs of the adders I N1 0 
to IN13. Each of them has two output terminals each 
having a different delay time. 

The signal outputted from a first output terminal 
TL10 of the delay circuit DL10 is multiplied by the pre- 
io determined coefficient by a multiplier KL1 0, and then the 
multiplied signal is supplied to a first input (i.e. , input for 
the left-side speaker) of a cross-talk canceler 2 via an 
adder AD10. On the other hand, the signal outputted 
from a second output terminal TR10 of the delay circuit 
is DL1 0 is multiplied by the predetermined coefficient by a 
multiplier KR10, and then the multiplied signal is sup- 
plied to a second input (i.e., input for the right-side 
speaker) of the cross-talk canceler 2 via adders AD12, 
AD13. 

Similarly, the signal outputted from a first terminal 
TL11 of the delay circuit DL11 is eventually supplied to 
the first input of the cross-talk canceler 2 via a multiplier 
KL1 1 and the adder AD1 0, while another signal output- 
ted from a second terminal TR11 of the delay circuit 
DL11 is eventually supplied to the second input of the 
cross-talk canceler 2 via a multiplier KR11 and the 
adders AD12, AD13. The signal outputted from a first 
terminal TL1 2 of the delay circuit DL1 2 is eventually sup- 
plied to the first input of the cross-talk canceler 2 via a 
multiplier KL1 2 and the adder AD11, AD1 0, while anoth- 
er signal outputted from a second terminal TR12 of the 
delay circuit DL12 is eventually supplied to the second 
input of the cross-talk canceler 2 via a multiplier KR12 
and the adder AD1 3. Lastly, the signal outputted from a 
first terminal TL1 3 of the delay circuit DL1 3 is eventually 
supplied to the first input of the cross-talk canceler 2 via 
a multiplier KL1 3 and the adders AD11 , AD10, while an- 
other signal outputted from a second terminal TR13 of 
the delay circuit DL13 is eventually supplied to the sec- 
ond input of the cross-talk canceler 2 via a multiplier 
KL13and the adder AD13. 

The above-mentioned cross-talk canceler 2 is de- 
signed to cancel the cross-talk sounds which are 
emerged when the person hears the sounds with his 
both ears. In other words, this is designed to eliminate 
the cross-talk phenomenon in which the right-side 
sound is entered into the left ear, while the left-side 
sound is entered into the right ear. Fig. 3(A) shows an 
example of the circuitry of this cross-talk canceler 2. This 
circuit is designed on the basis of the transfer function 
of head which is obtained through the study of the sound 
transmission between the human ears and dummy head 
(i.e., virtual simulation model of the human head). On 
the basis of the experimental values obtained through 
the transfer function of head, the study is made to com- 
pute the sound-arrival time differences between the left 
and right ears and the peak values of the impulse re- 
sponse of the transfer function. In response to these val- 
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ues, this circuitry performs the delay operations and 
weight functional caluculus. 

The observation is made on the model wherein both 
of the speakers SP(L), SP(R) are positioned apart from 
the person M by 1.5 m respectively and they are also s 
respectively arranged at the predetermined left-side and 
right-side positions each of which direction is deviated 
from the front direction of the person M by 45° . Since 
the foregoing transfer function of head of the person M 
is the symmetrical function, one of the speaker SP(L), 10 
SP(R) is sounded so as to actually measure the sound- 
arrival time difference between the left and right ears 
and the peak values of the impulse response. Herein, 
coefficients of multipliers and delay times of delay cir- 
cuits in the circuitry shown in Fig. 3(A) are determined is 
on the basis of the result of the measurement. For ex- 
ample, when the result of the measurement indicates 
that the left/right level difference is at 6dB (or 0.5) and 
the left/right time difference is at 200 u. s, the same co- 
efficient "-0.5" is applied to multipliers KL30, KR32, so 
while the same delay time 200 u, s is set to delay circuits 
DL30, DL32. Incidentally, the other circuit elements in 
Fig. 3(A), i.e., .delay circuits DL31 , DL33 and multipliers 
KL31 , KR33 configure the all-pass filter which is provid- 
ed to perform the phase matching. 25 

As shown in Fig. 1(A), the left and right output sig- 
nals of the cross-talk canceler 2 are amplified by an am- 
plifier 3 and then supplied to the left and right speakers 
SP(L), SP(R), from which the corresponding left/right 
sounds are produced. When listening to the sounds 30 
which are produced by means of the cross-talk canceler 
2, the cross talk is canceled, resulting that the clear 
sound separation between the left/right speakers is 
achieved. 

Next, the description will be given with respect to 35 
the functions of the delay circuits DL10-DL13. In case 
of the delay circuit DL10, the signal outputted from the 
terminal TR10 is multiplied by the predetermined coef- 
ficient in the multiplier KR10, and consequently, the mul- 
tiplied signal will be converted into the musical sound by 40 
the right speaker SP(R). On the other hand, the signal 
outputted from the terminal TL1 0 is multiplied by the pre- 
determined coefficient in the multiplier KL10, and con- 
sequently, the multiplied signal will be converted into the 
musical sound by the left speaker SP(L). In this case, 45 
the sound-image position is determined by two factors, 
i.e., the difference between the delay times of the 
sounds respectively produced from the right and left 
speakers, and the ratio between the tone volumes re- 
spectively applied to the left and right speakers. Since so 
the present embodiment can set the above-mentioned 
delay-time difference in addition to the above-men- 
tioned tone-volume ratio, the sound-image position can 
be set at certain position which is far from the speakers 
SP(L), SP(R) and which departs from the line connect- ss 
ing these speakers. In short, it is possible to set the 
sound-image position in the arbitrary space which de- 
parts from the linear space connecting the speakers. In 
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other words, the virtual speakers which are not actually 
existed are placed at the arbitrary spatial positions, so 
that the person can listen to the sounds which are virtu- 
ally produced from those positions. In the present em- 
bodiment, the delay circuit DL1 0 functions to set the vir- 
tual sound-producing position at VS10 (see Fig. 1(B)), 
which is called as the virtual speaker. 

Similarly, the other delay circuits DL11, DL12, DL13 
respectively correspond to the virtual speakers VS11, 
VS12, VS13 as shown in Fig. 1(B). As shown in Fig. 1 
(B), these virtual speakers VS10, VS11 , VS1 2, VS1 3 are 
respectively and roughly arranged along with a circular 
line which can be drawn about the performer. When 
drawing the center line between the performer (i.e., cir- 
cle center) and respective one of the virtual speakers 
VS10, VS11, VS12, VS1 3, there are formed four circular 
angles, 60° , 24° , 24° , 60° as shown in Fig. 1(B). 

Next, the description will be given with respect to 
the functions of the matrix controller MTR1. As de- 
scribed before, this matrix controller MTR1 is designed 
to control the connection relationship and arrangement 
relationship among the multipliers M1-M12, adders 
IN1 0-IN 1 3 and channels Ch1 0-Ch1 7. Such control indi- 
cates how to assign the signals of the channels 
Ch10-Ch17 to the virtual speakers VS10-VS13. Thus, 
the sound-image position of each channel Ch can be 
determined by the ratio of each channel-output signal 
applied to each virtual speaker. In other words, the pan- 
ning control is carried out on the virtual speakers 
VS10-VS1 3 respectively, thus controlling the sound-im- 
age position with respect to each channel. 

In the present embodiment as shown in Fig. 1(A), 
the allocation ratio of the each channel-output signal ap- 
plied to each virtual speaker is controlled by setting the 
coefficients of the multipliers M1-M12 as follows: 
CM1=0.75 (by being multiplied by this coefficient, the 
tone volume of the musical tone waveform signal is re- 
duced by 2.5dB), CM2=0.75, CM3=0.25 (by being mul- 
tiplied by this coefficient, the tone volume of the musical 
tone waveform signal is reduced by 12dB), CM4=0.75, 
CM5=0.625 (by being multiplied by this coefficient, the 
tone volume of the musical tone waveform signal is re- 
duced by 4.08dB), CM6=0.313 (which is equivalent to 
the reduction of 10.08dB in the tone volume of the mu- 
sical tone waveform signal), CM7=0.313, CM8=0.625, 
CM9=0.75, CM10=0.25, CM11=0.75, CM12-0.75. 

Fig. 2(A) shows another example of the arrange- 
ment and connection among the multipliers and adders 
under control of the matrix controller MTR1 . In this ex- 
ample, only two delay circuits DL10, DL13 are used for 
the virtual speakers. In short, as shown in Fig. 2(B), two 
virtual speakers VS10, VS13 are used for the production 
of the musical sounds. Herein, under control of the ma- 
trix controller MTR1 , some of the signals of the channels 
Ch10-Ch17 are adequately allocated to each of the 
adders I N 1 0, 1 N 1 3 so as to control the sound-image po- 
sitions. In this example, the coefficients of the multipliers 
M1-M14 are respectively set as follows: CM1=0.75, 



5 



• 



9 

CM2=0.75, CM3=0.313, CM4=0.625, CM5=0.375 (by 
being multiplied by this coefficient, the tone volume of 
the musical tone waveform signal is reduced by 8.5dB), 
CM6=0.5 (which is equivalent to the reduction of 6dB in 
the tone volume of the musical tone waveform signal), 
CM7=0.439 (which is equivalent to the reduction of 
7. 1 6dB in the tone volume of the musical tone waveform 
signal), CM8=0.439, CM9=0.5, CM1 0=0.375, 
CM1 1=0.625, CM1 2=0.31 3, CM1 3=0.75, CM1 4=0.75. 

(2) Operation 

Next, the description will be given with respect to 
the operation of the present embodiment. 

When the performer P plays the keyboard to per- 
form the music, the musical tone waveform signal is pro- 
duced in response to each of the keys depressed by the 
performer. Then, the musical tone waveform signals are 
respectively allocated to the channels on the basis of 
the predetermined tone-area allocation manner, so that 
these signals are eventually entered into the matrix con- 
troller MTR1. Assuming that the circuit elements of the 
matrix controller MTR1 are arranged and connected as 
shown in Fig. 1(A), the musical tone waveform signals 
are produced as the musical sounds from the virtual 
speakers VS10-VS1 3 in accordance with their tone ar- 
eas. 

The detailed explanation can be described as fol- 
lows. First of all, the musical tone waveform signals cor- 
responding to the tone area between the lowest-pitch 
note and C1 note (see Ch10) are produced as the mu- 
sical sounds from the virtual speaker VS10. In addition, 
the musical tone waveform signals corresponding to the 
tone area between the C#1 note and C2 note (see Ch11 ) 
are produced as the musical sounds from the virtual 
speakers VS12, VS1 0. However, due to the coefficients 
of the multipliers M2, M3, the sound-image positions 
corresponding to those notes are placed close to the vir- 
tual speaker VS10. More specifically, these sound-im- 
age positions are arranged on the line connecting the 
virtual speakers VS12, VS10, but they are also located 
close to the virtual speaker VS10. Further, the musical 
tone waveform signals corresponding to the tone area 
between the C#2 note to F2 note (see Ch12) are pro- 
duced as the musical sounds from the virtual speaker 
VS11. Similarly, the other musical tone waveform sig- 
nals corresponding to each the other tone areas (i.e., 
each of the other channels) are eventually produced as 
the musical sounds from the predetermined one or two 
virtual speakers at certain sound-image positions. Thus, 
the sound-image positions corresponding to the tone ar- 
eas which are respectively arranged from the lowest 
pitch to the highest pitch are sequentially arranged from 
the left-side position to the right-side position along with 
a circular line drawn about the performer P (see Fig. 1 
(B)). As a result, when the performer P sequentially de- 
press the keys from the lower pitch to the higher pitch, 
the sound-image positions are sequentially moved from 
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the left-side position to the right-side position along with 
the above-mentioned circular line. In short, it is possible 
to control the left/right and front/back positionings of the 
sound images. 

5 On the other hand, when the circuit elements of the 
matrix controller MTR1 are arranged and connected as 
shown in Fig. 2(A), the musical tone waveform signals 
of each tone area are eventually produced as the musi- 
cal sounds from one or both of the virtual speakers 

io VS1 0, VS1 3. Thus, the positioning control of the sound 
images are controlled on the line connecting these vir- 
tual speakers. In this case, the control of the front/back- 
side sound-broadened image is poor as compared to 
that of Fig. 1 (A). However, as comparing to the state 

15 where the musical sounds are merely produced from the 
left/right speakers SP(L), SP(R), this example can im- 
prove the control of the front/back-side sound broad- 
ened image. 

As described heretofore, the first embodiment is de- 
signed to change the allocation manner of the musical 
tone waveform signals by use of the matrix controller 
MTR 1 , therefore, it is possible to change over the control 
manner of the sound images with ease. 



Fig. 5 is a block diagram showing a modified exam- 
ple of the foregoing first embodiment, in which there are 
provided eight delay circuits DL50-DL57 used for 
30 emerging the virtual speakers. In Fig. 5, the illustration 
is partially omitted, so that there are also provided eight 
adders, in the matrix controller MTR1 , respectively cor- 
responding to the above-mentioned eight delay circuits 
DL50-DL57. According to the configuration of this mod- 
35 jfied example, eight virtual speakers are emerged, so 
that the musical tone waveform signals can be ade- 
quately allocated to these virtual speakers. Due to the 
provision of eight virtual speakers, it is possible to per- 
form the more precisely control on the sound-image po- 
40 sitions. 

[B] Second Embodiment 

Next, description will be given with respect to the 
45 second embodiment of the present invention by refer- 
ring to Fig. 6, wherein some parts corresponding to 
those of the foregoing first embodiment are omitted. 

In Fig. 6, numerals STR60-STR65 designate re- 
spective tone generators which are controlled by the 
50 MIDI signal (i.e., digital signal of which format is based 
on the standard for Musical Instruments Digital Inter- 
face). In short, one of the tone generators 
STR60-STR65 designated by the MIDI signal is activat- 
ed to produce the musical tone waveform signal. The 
55 outputs of these tone generators STR60-STR65 are re- 
spectively supplied to the delay circuits DL60-DL65 
which are used for forming the virtual speakers respec- 
tively. Then, the outputs of the delay circuits DL60-DL65 
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are multiplied by the predetermined coefficients respec- 
tively, so that some of the multiplied outputs are added 
together in adders VSR1-VSR4, VSL1-VSL4, of which 
addition results are supplied to the cross-talk canceler 2. 

According to the configuration of the above-men- 
tioned second embodiment, the output of each tone 
generator is produced as the musical sound from certain 
virtual speaker. Thus, when respectively connecting six 
strings of the guitar with six tone generators 
STR60-STR65, it is possible to well simulate the sound- 
producing manner of the guitar with respect to each 
string. The reason why such well simulation can be per- 
formed by the second embodiment is as follows: 

When the guitar is located close to the listener so 
that the strings are also located close to the ears of the 
listener, the listener can clearly discriminate the sepa- 
rate sound produced from each string of the guitar. How- 
ever, as the distance between the listener and guitar be- 
comes larger, the sound-separation image of each 
string of the guitar becomes weaker. Therefore, in the 
end, the sounds produced from all strings of the guitar 
will be heard as one overall sounds which are produced 
from one sound-production point. Thus, by adequately 
setting the delay times of the delay circuits DL60-DL65 
and the coefficients which are multiplied with the outputs 
of the delay circuits DL60-DL65, it is possible to offer 
the image of the distance by which the instrument is de- 
parted from the listener. 

In the meantime, it is possible to compute the dis- 
tance between the person and the virtual sound source 
which is embodied by the delay circuit as shown in Fig. 
7. Herein, V designates a radius of the head of the per- 
son M; a d n designates a distance between the sound 
source and the center of head; and "6" designates an 
angle which is formed between the sound source and 
the front-direction line of the head. In this case, it is pos- 
sible to compute distances B dr" and "dl" by the following 
equations, wherein B dr" designates a distance between 
the sound source and the right ear of the person, while 
"dl" designates a distance between the sound source 
and the left ear of the person. 

dr 2 = r 2 + d 2 -2rd*sin9 (1) 

dl 2 = r 2 -d 2 +2rd-sin6 (2) 

Thus, by computing these distances dr, dl with respect 
to each of the strings, it is possible to determine the fac- 
tors for designing the delay circuits DL60-DL65 respec- 
tively. 

Incidentally, in the aforementioned embodiments, it 
is possible for the user to arbitrarily set the connection 
pattern of the matrix controller MTR1 and the coefficient 
applied to each of the multipliers. Or, it is possible to 
store plural connection patterns and plural values for 
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each coefficient in advance, so that the user can arbi- 
trarily select one of them. 

[C] Third Embodiment 

Next, description will be given with respect to the 
third embodiment of the present invention, in which the 
sound-image position control apparatus 1 is applied to 
a game device 9, by referring to Figs. 8 to 10. 

Fig. 8 is a block diagram showing an electronic con- 
figuration of a game device 9. Herein, 10 designates a 
controller which controls the joy-stick unit, tracking-ball 
unit and several kinds of push-button switches (not 
shown) so that the operating states of them are sent to 
a control portion 11. The control portion 11 contains a 
central processing unit (i.e., CPU) and several kinds of 
interface circuits, whereas it is designed to execute the 
predetermined game programs stored in a program 
memory 12. Thus, the game is in progress, while the 
overall control of the game device is performed by the 
control portion 11 . In the progress of the game, a work- 
ing memory 1 3 is collecting and storing several kinds of 
data which are obtained through the execution of the 
game programs. In response to the game program to be 
executed, a visual image information memory 1 4 stores 
visual image data to be displayed, representing the in- 
formation of the visual images corresponding to charac- 
ter images C1, C2, C3 (given with the general numeral 
"C") and background images BG1, BG2, BG3 (given 
with the general numeral "BG"). These character imag- 
es may correspond to the visual images of person, au- 
tomobile, air plane, animal, or other kinds of objects. The 
above-mentioned visual image data are read out in the 
progress of the game, so that the corresponding visual 
image is displayed at the predetermined position of a 
display screen of a display unit 1 5 by the predetermined 
display size in response to the progress of the game. 

Next, a coordinate/sound- image-posit ion coeffi- 
cient conversion memory 16 stores parameters by 
which the display position of the character C in the dis- 
play unit 1 5 is located at the proper position correspond- 
ing to the sound-image position in the two-dimensional 
area. Fig. 9 shows a memory configuration of the above- 
mentioned coordinate/sound-image-position coefficient 
conversion memory 16. Fig. 10 shows a position rela- 
tionship between a player P of the game and the game 
device 9 in the two-dimensional area. The X-Y coordi- 
nates of the coordinate/sound-image-position coeffi- 
cient conversion memory 1 6 as shown in Fig. 9 may cor- 
respond to the X-Y coordinates of the display screen of 
the display unit 1 5. In Fig. 9, the output channel number 
CH of a sound source 17 and some of the coefficients 
CM1-CM12 which are used by the multipliers M1-M12 
in the sound-image position control apparatus 1 are 
stored at the memory area designated by the X-, Y-co- 
ordinate values which indicates the display position of 
the character C in the display unit 15. For example, at 
an area designated by "AR", a value "13" is stored as 
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the output channel number, while the other values "0.6" 
and "0.8" are also stored as the coefficients CM5, CM6 
used for the multipliers M5, M6 respectively. 

The X/Y coordinates of the coordinate/sound-im- 
age-position coefficient conversion memory 16 are set 5 
corresponding tothose of the actual two-dimensional ar- 
ea shown in Fig. 10. In other words, the display position 
of the character C in the display unit 1 5 corresponds to 
the actual two-dimensional position of the player as 
shown in Fig. 10. Thus, by adequately setting the pa- 
rameters, the sounds will be produced from the actual 
position corresponding to the display position of the 
character C. Incidentally, the memory area of the coor- 
dinate/sound-image-position coefficient conversion 
memory 1 6 is set larger than the display area of the dis- 
play unit 15. In this case, the proper channel number 
CH and some of the coefficients CM1-CM12 are mem- 
orized such that even if the character C is located at the 
coordinates of wh ich position cannot be displayed by the 
display unit 1 5, the sounds are produced from the actual 
position corresponding to the coordinates of the char- 
acter C. Moreover, the display position of the character 
C is controlled to be automatically changed in response 
to the progress of the game on the basis of the game 
programs stored in the program memory 12, or it is con- 
trolled to be changed in response to the manual opera- 
tion applied to the controller 10. 

Next, the sound source 17 has plural channels, 
used for the generation of the sounds, which are respec- 
tively operated in a time-division manner. Thus, in re- 
sponse to the instruction given from the control portion 
11, each channel produces a musical tone waveform 
signal. Such musical tone waveform signal is delivered 
to the predetermined one or some of eight channels 
Ch10-Ch17 of the sound-image position control appa- 
ratus 1 . Particularly, the musical tone waveform signal 
regarding to the character C is delivered to certain chan- 
nel Ch which is designated by the foregoing output 
channel number CH. As described before, this sound- 
image position control apparatus 1 has the electronic 
configuration as shown in Fig. 1(A), wherein the prede- 
termined coefficients CM1-CM12 are respectively ap- 
plied to the multipliers M1-M12 so as to control the 
sound-image position of each channel Ch when produc- 
ing the sounds from the speakers SP(L), SP(R). 

According to the electronic configurations as de- 
scribed heretofore, when the power is applied to the 
game device 9, the control portion 11 is activated to ex- 
ecute the programs stored in the program memory 1 2 
so as to progress the game. In response to the progress 
of the game, one of the background images BG1 , BG2, 
BG3 is selectively read from the visual image informa- 
tion memory 14 so that the selected background image 
is displayed on the display screen of the display unit 1 5. 
Similarly, one of the character images C1 , C2, C3 is se- 
lectively read out so that the selected character image 
is displayed in the display unit 15. Meanwhile, the con- 
trol portion 11 gives an instruction to the sound source 
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17 so as to produce the musical tone waveform signals 
corresponding to the background music in response to 
the progress of the game. In addition, the control portion 
11 also instructs the sound source 17 to produce the 
other musical tone waveform signals having the musical 
tone characteristics (such as the tone color, tone pitch, 
sound effects, etc.) corresponding to the character C. 
Moreover, the control portion 11 reads out the output 
channel number CH and coefficient CM (i.e., one or 
some of CM1-CM12) from the memory area of the co- 
ordinate/sound-image-position coefficient conversion 
memory 16 corresponding to the display position of the 
character C in the display unit 1 5, and then the read data 
are supplied to the sound source 17 and sound-image 
position control apparatus 1 respectively. In this case, 
the sound source 17 produces the musical tone wave- 
form signal corresponding to the character C, and this 
musical tone waveform signal is outputted to the sound- 
image position control apparatus 1 from the channel Ch 
which is designated by the output channel number CH. 
The other musical tone waveform signals are also out- 
putted to the sound-image position control apparatus 1 
from the corresponding channels respectively In the 
sound-image position control apparatus 1 , each of the 
coefficients CM read from the coordinate/sound- image- 
position coefficient conversion memory 16 is supplied 
to each of the multipliers M1-M12. Thus, the sound-im- 
age position of each channel is controlled to be fixed 
responsive to the coefficient CM, and consequently, the 
musical sounds are produced from the speakers SP(L), 
SP(R) at the fixed sound-image positions. 

When the player P intentionally operates the con- 
troller 10 to move the character C, the control portion 11 
is operated so that the display position of the character 
C displayed in the display unit 15 is moved by the dis- 
tance corresponding to the manual operation applied to 
the controller 10. In this case, new output channel 
number CH and coefficient CM are read from the mem- 
ory area of the coordinate/sound-image-position coeffi- 
cient conversion memory 16 corresponding to the new 
display position of the character C, and consequently, 
these data are supplied to the sound source 17 and 
sound-image position control apparatus 1 respectively. 
Thus, the actual sound-image position is also moved re- 
sponsive to the movement of the character C. 

According to the present embodiment, when the 
character C representing the visual image of the air 
plane is located outside of the display area of the display 
unit 1 5 and such character C is moved closer to the play- 
er P from his back, the character C is not actually dis- 
played on the display screen of the display unit 1 5. How- 
ever, since the foregoing coordinate/sound-image-posi- 
tion coefficient conversion memory 16 has the memory 
area which is larger than the display area of the display 
unit 15, the sounds corresponding to the character C 
are actually produced such that the sounds are coming 
closer to the player P from his back. As a result, the play- 
er P can recognize the existence and movement of the 
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air plane of which visual image is not actually displayed. 
This can offer a brand-new live-audio effect which can- 
not be obtained from the conventional game device sys- 
tem. 

Incidentally, the present embodiment is designed to 
manage the movement of the character C in the two- 
dimensional coordinate system. Of course, the present 
invention is not limited to it, so that the present embod- 
iment can be modified to manage the movement of the 
character C in the three-dimensional coordinate system. 
In such modification, number of the actual speakers are 
increased, and they are arranged in the three-dimen- 
sional space. 

In the present embodiment, the X/Y coordinates of 
the display unit 1 5 are set corresponding to those of the 
actual two-dimensional area. However, this embodi- 
ment can also modified to simulate the game of the au- 
tomobile race. In this case, only the character C which 
is displayed in front of the player P is displayed in the 
display unit 1 5 by matching the visual range of the player 
P with the display area of the display unit 15. 

[D] Fourth Embodiment 

Next, the description will be given with respect to 
the fourth embodiment of the present invention, wherein 
the sound-image position control apparatus is modified 
to be applied to the movie system, video game device 
(or television game device) or so-called CD-I system in 
which the sound-image position is controlled responsive 
to the video image. 

Before describing the fourth embodiment in detail 
in conjunction with Figs. 11 to 1 3, the description will be 
given with respect to the background of the fourth em- 
bodiment by referring to Figs. 14 and 15. 

First of all, the so-called binaural technique is 
known as the technique which controls and fixes the 
sound-image position in the three-dimensional space. 
According to the known technique, the sounds are re- 
corded by use of the microphones which are located 
within the ears of the foregoing dummy head, so that the 
recorded sounds are reproduced by use of the head- 
phone set so as to recognize the sound-image position 
which is fixed at the predetermined position in the three- 
dimensional space. Recently, some attempts are made 
to simulate the tone area which is formed in accordance 
with the shape of the dummy head. In other words, by 
simulating the transfer function of the sounds which are 
transmitted in the three-dimensional space by use of the 
digital signal processing technique, the sound-image 
position is controlled to be fixed in the three-dimensional 
space. 

The coordinate system of the above-mentioned 
three dimensional space can be defined by use of the 
illustration of Fig. 14. In Fig. 14, V designates a distance 
from the origin "O"; $ designates an azimuth angle with 
respect to the horizontal direction which starts from the 
origin "O"; 0 designates an elevation angle with respect 



to the horizontal area containing the origin "O", thus, the 
three-dimensional space can be defined by the polar co- 
ordinates in the space. When locating the listener or 
dummy head at the origin O, its front direction can be 
5 defined as <j> =0, whereas its left-side direction is defined 
by (J> >0 and its right-side direction is defined by (|><0. In 
addition, the upper direction is defined by 9 >0. 

As a model which controls and fixes the sound-im- 
age position in the three-dimensional space by use of 
io the digital signal processing technique, the dummy head 
is located at the origin 0 and then the impulse signal is 
produced from the predetermined point A, for example. 
Then, the responding sounds corresponding to the im- 
pulse signal are sensed by the microphones which are 
15 respectively located within the ears of the dummy head. 
These sensed sounds are converted into the digital sig- 
nals which are recorded by some recording medium. 
These digital signals represent two impulse-response 
data respectively corresponding to the sounds picked 
20 up by the left-side and right-side ears of the dummy 
head. These two impulse-response data are converted 
into the coefficients, by which two finite-impulse re- 
sponse digital filters (hereinafter, simply referred to as 
Fl R filters) are respectively given. In this case, the audio 
25 signal of which sound-image position is not fixed is de- 
livered to two FIR filters, through which two digital out- 
puts are obtained as the left/right audio signals. These 
left/right audio signals are applied to left/right inputs of 
the headphone set, so that the listener can hear the ster- 
eo eophonic sounds from this headphone set as if those 
sounds are produced from the point A. By changing this 
point A and measuring the impulse response, it is pos- 
sible to obtain the other coefficients for the FIR filters. 
In other words, by locating the point A at the desirable 
35 position, it is possible to obtain the coefficients for the 
FIR filters, by which the sound-image position can be 
fixed at the desirable position. The above-mentioned 
technique offers an effect by which the three-dimension- 
al sound-image position is determined by use of the 
40 sound-reproduction system of the headphone set. The 
same effect can be embodied by use of the so-called 
two-speaker sound-reproduction system in which two 
speakers are located at the predetermined front posi- 
tions of the listener, which is called a cross-talk cance- 
ls ling technique. 

According to the cross-talk canceling technique, the 
sounds are reproduced as if they are produced from cer- 
tain position (i.e., position of the foregoing virtual speak- 
er) at which the actual speaker is not located. Herein, 
50 two FIR filters are required when locating one virtual 
speaker, hereinafter, a set of two FIR filters will be called 
as a sound-directional device. 

Fig. 15 is a block diagram showing an example of 
the virtual-speaker circuitry which employs the above- 
55 mentioned sound-directional device. In Fig. 1 5, 1 02-1 04 
designate sound-directional devices, each of which con- 
tains two FIR filters. This drawing only illustrates three 
sound-directional devices 102-104, however, there are 
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actually provided several hundreds of the sound-direc- 
tional devices. Thus, it is possible to locate hundreds of 
virtual speakers in a close-tight manner with respect to 
all of the directions of the polar-coordinate system. 
These virtual speakers are not merely arranged along 
with a spherical surface with respect to the same dis- 
tance r, but they are also arranged in a perspective man- 
ner with respect to different distances r. A selector 101 
selectively delivers the input signal to one of the sound- 
directional devices such that the sounds will be pro- 
duced from the predetermined one of the virtual speak- 
ers, thus controlling and fixing the sound-image position 
in the three-dimensional space. Incidentally, adders 
105, 106 output their addition results as the left/right au- 
dio outputs respectively. 

The above-mentioned example can be modified 
such that one sound-directional device is not fixed cor- 
responding to one direction of producing the sound. In 
other words, by changing the coefficients of the FIR fil- 
ters contained in one sound-directional device, it is pos- 
sible to move the sound-image position by use of only 
one sound-directional device. 

In the meantime, some movie theater employs so- 
called surround acoustic technique which uses four or 
more speakers. Therefore, the sounds are produced 
from one or some speakers in response to the video im- 
age. 

When embodying such surround acoustic tech- 
nique by use of the former virtual-speaker system pro- 
viding hundreds of sound-directional devices, it is nec- 
essary to provide hundreds of FIR filters, which enlarges 
the scale of the system so that the cost of the system 
will be eventually raised up. Even in the case of the latter 
system which provides only one sound-directional de- 
vice, it is necessary to provide hundreds of coefficients 
used for the FIR filter, which is not realistic. Because, it 
is very difficult to control or change so many number of 
coefficients in a real-time manner. Further, when em- 
bodying the foregoing surround acoustic technique in 
the movie theater, it is necessary to provide a plenty of 
amplifiers and speakers, which eventually raises the 
cost of the facilities. 

(a) Configuration of Fourth Embodiment 

Next, the detailed description will be given with re- 
spect to the fourth embodiment of the present invention. 
Fig. 11 is a block diagram showing the whole configura- 
tion of the video game system. Herein, a game device 
21 is designed to produce a video signal VS, a left-side 
musical tone signal ML, a right-side musical tone signal 
MR, a sound effect signal EFS, a panning signal PS and 
a scene-identification signal SCS. When receiving the 
sound effect signal EFS, panning signal PS and scene- 
identification signal SCS, a sound-image position con- 
trol apparatus 22 imparts the fixed sound image to the 
sound effect signal EFS, thus producing two signals EF- 
SL, EFSR. Then, an adder 25 adds the signals EFSR 



and MR together, while an adder 26 adds the signals 
EFSL and ML together. The results of the additions re- 
spectively performed by the adders 25, 26 are supplied 
to an amplifier 24. The amplifier 24 amplifies these sig- 
5 nals so as to respectively output the amplified signals to 
left/right loud-speakers (represented by 43, 44 in Fig. 
1 3). In the meantime, the video signal VS is supplied to 
a video device 23, so that the video image is displayed 
for the person. 

io The game device 21 is configured as the known vid- 
eo game device which is designed such that responsive 
to the manipulation of the player of the game, the scene 
displayed responsive to the video signal VS is changed 
or the position of the character image is moved. During 

'5 the game, the musical tone signals ML, MR are output- 
ted so as to playback the background music. In addition, 
to this background music, the other sounds are also pro- 
duced. For example, the sounds corresponding to the 
character image which is moved responsive to the ma- 

20 nipulation of the player, or the other sounds correspond- 
ing to the other character images which are automati- 
cally moved under control of the control unit built in the 
game device 21 are produced by the sound effect signal 
EFS. In case of the game of the automobile race, the 

25 engine sounds of the automobiles are automatically pro- 
duced. 

The scene-identification signal SCS is used for de- 
termining the position of the virtual speaker in accord- 
ance with the scene. Every time the scene is changed, 

30 this scene-identification signal SCS is produced as the 
information representing the changed scene. Such 
scene-identification signal SCS is stored in advance 
within a memory unit (not shown) which is built in the 
game device 21. More specifically, this signal is stored 

35 at the predetermined area adjacent to the area storing 
the data representing the background image with re- 
spect to each scene of the game. Thus, when the scene 
is changed, this signal is simultaneously read out. 
On the other hand, the panning signal PS repre- 

40 sents certain position which is located between two vir- 
tual speakers. By varying the value of this panning sig- 
nal PS between "0" and "1", it is possible to freely 
change the sound-image position corresponding to the 
sound produced responsive to the sound effect signal 

45 EFS between two virtual speakers. In the present em- 
bodiment, the programs of the game contain the oper- 
ation routine for the panning signal PS, by which the 
panning signal PS is computed on the basis of the 
scene-identification signal SCS and the displayed posi- 

50 tion of the character image corresponding to the sound 
effect signal EFS. Of course, such computation of the 
panning signal PS can be omitted, so that in response 
to the position of the character, the game device 21 au- 
tomatically reads out the panning signal PS which is 

55 stored in advance in the memory unit. Incidentally, the 
present embodiment is designed such that two virtual 
speakers are emerged, which will be described later in 
detail. 
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Fig. 12 is a block diagram showing an internal con- 
figuration of the sound-image position control apparatus 
22. Herein, a control portion 31 is configured as the cen- 
tral processing unit (i.e., CPU), which performs the over- 
all control on this apparatus 22. This control portion 31 
receives the foregoing scene-identification signal SCS 
and panning signal PS. A coefficient memory 32 stores 
the coefficients of the FIR filters. As described before, 
the impulse response is measured with respect to the 
virtual speaker which is located at the desirable position, 
so that the above-mentioned coefficients are deter- 
mined on the basis of the result of the measurement. In 
order to locate the virtual speaker at the optimum posi- 
tion corresponding to the scene of the game, the coef- 
ficients for the FIR filters are computed in advance with 
respect to several positions of the virtual speaker, and 
consequently, these coefficients are stored at the ad- 
dresses of the memory unit corresponding to the scene- 
identification signal SCS. As described before, each of 
sound-directional devices 33, 34 is configured by two 
FIR filters. The coefficient applied to the FIR filter can 
be changed by the coefficient data given from the control 
portion 31 . 

In response to the scene-identification signal SCS, 
the control portion 11 reads out the coefficient data, re- 
spectively corresponding to the virtual speakers L, R, 
from the coefficient memory 32, and consequently, the 
read coefficient data are respectively supplied to the 
sound-directional devices 33, 34. When receiving the 
coefficients, each of the sound-directional devices 33, 
34 performs the predetermined signal processing on the 
input signal of the FIR filters, thus locating the virtual 
speaker at the optimum position corresponding to the 
scene-identification signal SCS. 

The sound effect signal EFS is allocated to the 
sound-directional devices 33, 34 via multipliers 35, 36 
respectively. These multipliers 35, 36 also receive the 
multiplication coefficients respectively corresponding to 
the values "PS", M 1 -PS" from the control portion. Herein, 
the value "PS" represents the value of the panning sig- 
nal PS, while the value "1 -PS" represents the one's com- 
plement of the panning signal PS. The outputs of first 
FIR filters in the sound-directional devices 33, 34 are 
added together by an adder 37, while the other outputs 
of second FIR filters in the sound-directional devices 33, 
34 are added together by another adder 38. Therefore, 
these adders 37, 38 output their addition results as sig- 
nals for the speakers 43, 44 respectively. These signals 
are supplied to a cross-talk canceler 39. 

The cross-talk canceler 39 is provided to cancel the 
cross-talk component included in the sounds. For ex- 
ample, the cross-talk phenomenon must be occurred 
when producing the sounds from the speakers 43, 44 in 
Fig. 13. Due to this cross-talk phenomenon, the sound 
component produced from the left-side speaker affects 
the sound which-is produced from the right-side speaker 
for the right ear of the listener, while the sound compo- 
nent produced from the right-side speaker affects the 



sound which is produced from the left-side speaker for 
the left ear of the listener. Thus, in order to cancel the 
above-mentioned cross-talk components, the cross-talk 
canceler 39 performs the convolution process by use of 
5 the phase-inverted signal having the phase which is in- 
verse to that of the cross-talk component. Under oper- 
ation of this cross-talk canceler 39, the outputs of the 
sound-directional device 33 are converted into the 
sounds which are roughly heard by the left ear only from 
10 the left-side speaker, while the outputs of the sound-di- 
rectional device 34 are converted into the sounds which 
are roughly heard by the right ear only from the right- 
side speaker. Such sound allocation can roughly em- 
body the situation in which the listener hears the sounds 
15 by use of the headphone set. 

Meanwhile, the cross-talk canceler 39 receives a 
cross-talk bypass signal BP from the control portion 31 . 
This cross-talk bypass signal BP is automatically pro- 
duced by the control portion 31 when inserting the head- 
20 phone plug into the headphone jack (not shown). When 
the headphone plug is not inserted, the cross-talk by- 
pass signal BP is turned off, so that the sounds are re- 
produced from two speakers while canceling the cross- 
talk components as described before. On the other 
25 hand, when the headphone plug is inserted, the cross- 
talk canceling operation is omitted, so that the signals 
are supplied to the headphone set from which the 
sounds are reproduced. 

Next, the description will be given with respect to 
30 the method how to control and fix the sound-image po- 
sition by the panning signal PS. When the value of the 
panning signal PS is equal to zero, the foregoing sound 
effect signal EFS is supplied to the sound-directional de- 
vice 34 only. Thus, the sound-image position is fixed at 
35 the position of the virtual speaker (i.e., position of the 
speaker 45 in Fig. 13) which is located by the sound- 
directional device 34. On the other hand, when the value 
of the panning signal PS is at "1 n , the sound effect signal 
EFS is supplied to the sound-directional device 33 only, 
40 and consequently, the sound-image position is fixed at 
the position of the virtual speaker (i.e., position of a 
speaker 46) which is located by the sound-directional 
device 33. When the value of the panning signal PS is 
set at a point between "0 n and "1 ", the sound-image po- 
45 sition is fixed at an interior-division point corresponding 
to the panning signal PS between the virtual speakers 
45, 46. 

(b) Operation of Fourth Embodiment 

50 

Next, description will be given with respect to the 
operation of the fourth embodiment by referring to Fig. 
13. In Fig. 13, a player 41 is positioned at the center, 
whereas the left-side speaker 43 is located at the left/ 
55 front-side position from the player 41 which is defined 
by <J> =45° , 6 =0° , r=1 .5m, while the right-side speaker 
44 is located at the right/front-side position from the 
player 41 which is defined by <J> =-45° , 6 =0° , r=1.5m. 
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In front of the player 41 , there is located a display screen 
42 of the video device 23. In the present embodiment, 
this display screen 42 has a flat-plate-like shape, how- 
ever, it is possible to form this screen by the curved sur- 
face which surrounds the player 41 . 5 

For example, the player 41 plays the game and the 
duel scene of the Western is displayed. In this case, the 
game device 21 outputs the scene-identification signal 
SCS to the control portion 31 in the sound-image posi- 
tion control apparatus 22, wherein this scene-identifica- 
tion signal SCS has the predetermined scene-identify- 
ing value, e.g., four-bit data "0111". Then, the control 
portion 31 reads out coefficient data CL, corresponding 
to the scene-identification signal SCS, from the coeffi- 
cient memory 32, wherein this coefficient data CL rep- 
resents the coefficient for the FIR filter which corre- 
sponds to the position of the left-side virtual speaker 45 
(defined by <J> =85° , 6=0° , r=3.5m). This coefficient data 
CL is supplied to the sound-directional device 33. In ad- 
dition, the control portion 31 also read out another co- 
efficient data CR representing the coefficient for the FIR 
filter which corresponds to the position of the right/up- 
per-side virtual speaker 46 (defined by <|> =-40° , 6 =65° , 
r=15.0m). This coefficient data CR is supplied to the 
sound-directional device 34. Thus, the virtual speakers 
45, 46 are located at their respective positions as shown 
in Fig. 13. 

The game device 21 produces the musical tone sig- 
nals ML, MR which are sent to the speakers 43, 44 via 
the adders 25, 26 and amplifier 24, whereas the music 
which is suitable for the duel scene is reproduced, while 
the other background sounds such as the wind sounds 
are also reproduced, regardless of the sound-image po- 
sition control. I n response to a shot action of a gunf ighter 
which is the displayed image and plays an enemy role 
for the player 41 in the gunfight game, the sound effect 
signal EFS representing a gunshot sound is supplied to 
the sound-image position control apparatus 22. In this 
case, if the value of the panning signal PS is equal to 
zero, the gunshot is merely sounded from the position 
of the virtual speaker R46. Such sound effect corre- 
sponds to the scene in which the gunfighter shoots a 
gun by aiming at the player 41 from the second floor of 
the saloon. On the other hand, if the value of the panning 
signal PS is equal to "1 n , the gunshot may be sounded 
in the scene in which the gunfighter is placed at the left- 
side position very close to the player 41 and then the 
gunfighter shoots a gun at the player 41 . If the value of 
the panning signal PS is set at certain value between 
"0" and "1", the gunfighter is placed at certain interior- 
division point on the line connected between the virtual 
speakers 45, 46, and then the gunshot is sounded. 

The game device 21 is designed such that even in 
the same duel scene of the Western, every time the po- 
sition of the enemy is changed, new scene-identification 
signal SCS (having a new binary value such as "1010") 
is produced and outputted to the sound-image position 
control apparatus 22. In other words, the change of the 



position of the enemy is dealt as the change of the 
scene. Thus, the virtual speakers will be located again 
in response to the new scene. 

Other than the above-mentioned Western game, 
the game device 21 can also play the automobile race 
game. Herein, the game device 21 outputs a new scene- 
identification signal SCS (having a binary value such as 
■0010°), by which the control portion 31 reads out two 
coefficient data respectively corresponding to the right/ 
front-side virtual speaker and right/back-side virtual 
speaker. These coefficient data are respectively sup- 
plied to the sound-directional devices 33, 34. In this 
case, the foregoing signals ML, MR represent the back- 
ground music and the engine sounds of the automobile 
to be driven by the player 41 . Further, the foregoing sig- 
nal EFS represents the engine sounds of the other au- 
tomobiles which will be running in the race field as the 
displayed images. On the basis of the foregoing opera- 
tion routine, the panning signal PS is computed and re- 
newed in response to the position relationship between 
the player's automobile and the other automobiles. If an- 
other automobile is running faster than the player's au- 
tomobile so that another automobile will get ahead of 
the player's automobile, the value of the panning signal 
PS is controlled to be gradually increased from "0 n to 
"1 Thus, in response to the scene in which another au- 
tomobile gets ahead of the player's automobile, the 
sound-image position of the engine sound of another 
automobile is controlled to be gradually moved ahead. 

As described above, the fourth embodiment is ap- 
plied to the game device. However, it is possible to mod- 
ify the present embodiment such that the sound-image 
position control is performed in response to the video 
scene played by the video disk player Or, it is possible 
to apply the present embodiment to the CD-I system. In 
this case, the foregoing scene-identification signal SCS 
and panning signal PS can be recorded at the sub-code 
track provided for the audio signal. 

Further, the present embodiment uses two sound- 
directional devices, however, it is possible to modify the 
present embodiment such that three or four sound-di- 
rectional devices are provided to cope with more com- 
plicated video scenes. In this case, the complicated con- 
trol must be performed on the panning signal PS. How- 
ever, it is not necessary to provide hundreds of sound- 
directional devices, or it is not necessary to simultane- 
ously change hundreds of coefficients for the FIR filters. 

Moreover, the sound-directional device of the 
present embodiment is configured by the FIR filters, 
however, this device can be configured by the infinite- 
impulse response digital filters (i.e., MR filters). For ex- 
ample, the so-called notch filter is useful when fixing the 
sound-image position with respect to the elevation-an- 
gle direction. Further, it is also known that the band-pass 
filter controlling the specific frequency-band is useful 
when controlling the sound-image position with respect 
to the front/back direction. When embodying such filter 
by use of the MR filters, the fixing degree of the sound- 
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image position may be reduced as compared to the FIR 
filters. However, the II R filter has a simple configuration 
as compared to the FIR filter, so that the number of the 
coefficients can be reduced. In short, the MR filter is ad- 
vantageous in that the controlling can be made easily. 

Lastly, this invention may be practiced or embodied 
in still other ways without departing from the character 
as defined by the appended claims. Therefore, the pre- 
ferred embodiments described herein are illustrative 
and not restrictive, the scope of the invention being in- 
dicated by the appended claims and all variations which 
come within the meaning of the claims are intended to 
be embraced therein. 



Claims 

1 . A sound image position control apparatus compris- 
ing: 

a sound signal input means (CH10-17) for in- 
putting sound signals; 

at least two real speakers (SP(L), SP(R)) for 
outputting output sound signals; 
a plurality of virtual speaker forming means 
(DL10-13, KL10-13, KR10-13) coupled to the 
real speakers each of the plurality of virtual 
speaker forming means processing a sound 
signal input thereto so that a sound correspond- 
ing to the processed sound signal and gener- 
ated by at least two of the real speakers is lo- 
calised at a desired position, in a listening 
space wherein therefore, each of the plurality 
of virtual speaker forming means virtually forms 
a virtual speaker at the desired position, where- 
in each of the plurality of virtual speaker forming 
means generates right-side and left-side sig- 
nals by processing the signal input thereto; 
a mixing means (AD10-1 3) for mixing the right- 
side signals output by the at least two of the 
virtual speaker forming means and then provid- 
ing the mixed right-side signals to one of at 
least two of the real speakers and for mixing 
the left-side signals output by the at least two 
of the virtual speaker forming means and then 
providing the mixed left-side signals to the oth- 
er of the at least two of the real speakers; 
a distributing means (MTR1 ; 35, 36) for receiv- 
ing a sound signal from the sound signal input 
means and then distributing it among at least 
two of the virtual speaker forming means at a 
distribution ratio; 

wherein a sound, corresponding to the received 
sound signal, generated by the at least two of 
the real speakers is localised at a position be- 
tween at least two virtual speakers, formed by 
the at least two of the virtual speaker forming 
means, in accordance with the distribution ra- 



tio. 

2. A sound image position control apparatus accord- 
ing to claim 1 , wherein the distribution ratio is de- 

5 termined in accordance with a frequency character- 
istic of the sound signal. 

3. The sound image position control apparatus of 
claim 1, further comprising a crosstalk cancellation 

10 circuit coupled to the mixing means and the real 
speakers. 

4. The sound image position control apparatus of 
claim 1 , wherein the sound signal input means corn- 
's prises a tone generator controlled by a MIDI signal. 

5. The sound image position control apparatus of 
claim 1, wherein the distributing means includes a 
matrix controller containing plural multipliers and 

20 plural adders in which a connection pattern is 
changed to accommodate a change in a signal dis- 
tributing protocol. 

6. A sound image position control apparatus accord- 
25 jng to claim 1 , wherein each of the virtual-speaker 

forming means comprises: 

a delay circuit (DL10-13) having two delay 
times, the delay circuit delaying a sound signal 
30 input thereto by each of the delay times so as 

to output two delayed signals; and 
an allocation ratio applying means (KL10-13, 
KR10-1 3) for applying a predetermined alloca- 
tion ratio to the delayed signals, thus allocating 
35 the delayed signals to the at least two of the 

real speakers as the right-side and left-side sig- 
nals. 

7. A sound image position control apparatus accord- 
40 jng to claim 1 , wherein the plurality of virtual speaker 

forming means comprise a finite-impulse-response 
digital filter. 

8. A sound-image position control apparatus accord- 
45 jng to claim 1, wherein the distributing means in- 
cludes distribution control means which controls the 
distribution rate to change a localization position of 
the sound between the positions of the at least two 
virtual speakers. 

50 

9. A sound image position control apparatus accord- 
ing to any of the preceding claims further compris- 
ing: 

55 display means (15) for displaying a visual im- 

age corresponding to a video signal; 
video signal producing means (21 in Fig. 12), 
included by the sound signal input means, for 
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producing a video signal (VS), a sound signal 
(EFS) and a scene-identification signal (SCS), 
the scene-identification signal corresponding 
to each scene of a display image on the display 
means; and 

control means (31) for controlling at least two 
of the virtual speaker forming means (33, 34) 
in accordance with the scene-identification sig- 
nal so that positions of at least two of the virtual 
speakers are controlled in response to the 
scene-identification signal. 

10. A sound image position control apparatus accord- 
ing to claim 9, wherein the video signal producing 
means (21) also produces a panning signal (PS) 
corresponding to the distribution ratio, and then the 
control means controls the distributing means (35, 
36) in accordance with the panning signal so that a 
position of the sound signal localized between the 
at least two of the virtual speakers is moved in re- 
sponse to the panning signal. 

11. A sound image position control apparatus accord- 
ing to anyone of claims 1 through 8 further compris- 
ing: 

a display means (15) for displaying a predeter- 
mined animated image on a display screen 
thereof, said animated image corresponding to 
the sound to be virtually produced by the at 
least two virtual speakers, wherein a display 
position of said animated image corresponds to 
a position of a sound to be localized by the at 
least two virtual speakers so that the position 
of the sound to be localized corresponding to 
said animated image is moved in accordance 
with a movement of said animated image on the 
display screen of said display means. 

12. A sound-image position control apparatus as de- 
fined in claim 9 wherein said virtual speaker forming 
means includes at least two virtual-speaker position 
control means (33, 34) which respectively perform 
predetermined signal processings corresponding to 
said scene -identification signal on said audio signal 
so as to form at least two virtual speakers by which 
the sound image corresponding to said audio signal 
is formed. 

13. A sound-image position control apparatus as de- 
fined in claim 1 2 wherein an audio/video information 
producing means is provided producing a panning 
signal (PS) by which the sound image position is 
located at a certain interior-division point in linear 
space connected between said virtual speakers. 



Patentanspruche 

1. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle, welche Vorrichtung folgendes auf- 

5 weist: 

Tonsignaleingabemittel (CH10-17) zurEingabe 
von Tonsignalen; 

wenigstens zwei reale Lautsprecher (SP(L), SP 
10 (R)) zur Ausgabe von Ausgangstonsignalen; 

eine Vielzahl von Virtuell-Lautsprecher-Bil- 
dungsmitteln (DL10-13, KL10-13, KR10-13), 
die an die realen Lautsprecher gekoppelt sind, 
wobei jedes aus der Vielzahl der Virtuell-Laut- 
*5 sprecher-Bildungsmittel ein ihm zugefuhrtes 

Tonsignal verarbeitet, so daG ein Ton, der dem 
verarbeiteten Tonsignal entspricht und von we- 
nigstens zwei realen Lautsprechern erzeugt 
wird, an einer gewunschten Position in einem 
akustischen Raum lokalisiert ist, wobei jedes 
aus der Vielzahl der Virtuell-Lautsprecher-Bil- 
dungsmittel einen virtuellen Lautsprecher an 
der gewunschten Position bildet und jedes aus 
der Vielzahl der Virtuell-Lautsprecher-Bil- 
dungsmittel rechtseitige und linksseitige Signa- 
le durch Verarbeitung der an sie eingegebenen 
Signale erzeugt; 

Mischmittel (AD10-13), die die rechtsseitigen 
Signale, die von wenigstens zwei der Virtuell- 
Lautsprecher-Bildungsmittel ausgegeben wer- 
den, mischen und dann die gemischten recht- 
seitigen Signale einem der wenigstens zwei 
realen Lautsprecher zufuhren, und die die 
linksseitigen Signale, die von wenigstens zwei 
der Virtuell-Lautsprecher-Bildungsmittel aus- 
gegeben werden, mischen und dann die ge- 
mischten linksseitigen Signale dem anderen 
der wenigstens zwei realen Lautsprecher zu- 
fuhren; 

Verteilermittel (MTR1 ; 35, 36), die ein Tonsignal 
von den Tonsignaleingabemitteln empfangen 
und dieses dann auf die wenigstens zwei Vir- 
tuell-Lautsprecher-Bildungsmittel mit einem 
Verteilungsverhaltnis verteilen; 
wobei der von wenigstens zwei der realen Laut- 
sprecher erzeugte Ton, der dem empfangenen 
Tonsignal entspricht, bei einer Position zwi- 
schen wenigstens zwei virtuellen Lautspre- 
chern, die durch die wenigstens zwei Virtuell- 
Lautsprecher-Bildungsmittel gebildet werden, 
entsprechend dem Verteilungsverhaltnis loka- 
lisiert wird. 

2. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 1 , bei welcher Vor- 
richtung das Verteilungsverhaltnis entsprechend ei- 
ner Frequenzcharakteristik des Tonsignals be- 
stimmt wird. 
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3. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 1, welche Vorrich- 
tung auGerdem eine Ubersprechunterdruckungs- 
schaltung aufweist, die an die Mischmittel und die 
realen Lautsprecher gekoppelt ist. s 

4. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 1 , bei welcher Vor- 
richtung die Tonsignaleingabemittel einen durch ein 
MIDI-Signal gesteuerten Tongenerator aufweisen. 10 

5. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 1, bei welcher Vor- 
richtung die Verteilermittel eine Matrixsteuereinheit 
enthalten, die mehrere Multiplizierer und mehrere is 
Addierer aufweist, in der ein Verbindungsmuster in 
Anpassung an die Anderung eines Signalvertei- 
lungsprotokolls geandert wird. 

6. Vorrichtung zur Steuerung der Position des Bildes 20 
einer Tonquelle nach Anspruch 1, bei welcher Vor- 
richtung jedes der Vlrtuell-Lautsprecher-Bildungs- 
mittel folgendes aufweist: 

eine Verzogerungsschaltung (DL10-13) mit 2s 
zwei Verzogerungszeiten, wobei die Verzoge- 
rungsschaltung ein ihreingegebenesTonsignal 
urn jede der Verzogerungszeiten so verzogert, 
daG zwei verzogerte Signale ausgegeben wer- 
den; und 30 
Mittel zum Zuweisen eines Zuordnungsverhalt- 
nisses (KL1 0-1 3, KR1 0-1 3), urn ein vorgegebe- 
nes Zuordnungsverhaltnis den verzogerten Si- 
gnalen zuzuweisen, wobei so die verzogerten 
Signale den wenigstens zwei realen Lautspre- 35 
chern als rechtsseitige und linksseitige Signale 
zugeordnet werden. 

7. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 1 , bei welcher Vor- 40 
richtung die Vielzahl der Virtuell-lautsprecher-Bil- 
dungsmittel ein nichtrekursives Digitalfilter aufwei- 
sen. 

8. Vorrichtung zur Steuerung der Position des Bildes *s 
einer Tonquelle nach Anspruch 1 , bei welcher Vor- 
richtung die Verteilermittel Verteilungssteuermittel 
enthalten, die das Verteilungsverhaltnis steuern, 

urn eine Lokalisationsposition des Tons zwischen 
den Positionen der wenigstens zwei virtuellen Laut- so 
sprecher zu andern. 

9. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach irgendeinem der vorherge- 
henden Anspruche, welche Vorrichtung auGerdem ss 
folgendes aufweist: 

Anzeigemittel (15) zur Anzeige eines einem Vi- 
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deosignal entsprechenden visuellen Bildes; 
Videosignalerzeugungsmittel (21 in Fig. 12), 
die in den Tonsignaleingabemitteln enthalten 
sind, zur Erzeugung eines Videosignals (VS), 
eines Tonsignals (EFS) und eines Szeneniden- 
tifizierungssignals (SCS), wobei das Szenen- 
identifizierungssignal jeder Szene eines Anzei- 
gebildes auf den Anzeigemitteln entspricht; 
und 

Steuermittel (31) zur Steuerung von wenig- 
stens zwei der Virtuell-Lautsprecher-Bildungs- 
mittel (33, 34) entsprechend dem Szeneniden- 
tifizierungssignal, so daG Positionen von we- 
nigstens zwei der virtuellen Lautsprecher im 
Ansprechen auf das Szenenidentifizierungssi- 
gnal gesteuert werden. 

10. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 9, bei welcher Vor- 
richtung die Videosignalerzeugungsmittel (21 ) auch 
ein Panoramasignal (PS) erzeugen, das dem Ver- 
teilungsverhaltnis entspricht, wobei die Steuermit- 
tel die Verteilermittel (35, 36) entsprechend dem 
Panoramasignal steuern, so daG eine zwischen den 
wenigstens zwei virtuellen Lautsprechern lokali- 
sierte Position des Tonsignals im Ansprechen auf 
das Panoramasignal bewegt wird. 

11. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach irgendeinem der AnsprOche 1 
bis 8, welche Vorrichtung auGerdem folgendes auf- 
weist: 

Anzeigemittel (15) zur Anzeige eines vorgege- 
benen beweglichen Bildes auf deren Anzeige- 
bildschirm, wobei das bewegliche Bild dem Ton 
entspricht, der von den wenigstens zwei virtu- 
ellen Lautsprechern virtue!! erzeugt werden 
soli, wobei eine Anzeigeposition des bewegli- 
chen Bildes einer Position eines Tons ent- 
spricht, der von den wenigstens zwei virtuellen 
Lautsprechern so lokalisiert werden soil, daG 
die zu lokalisierende Position des Tons, der 
dem beweglichen Bild entspricht, entspre- 
chend einer Bewegung des beweglichen Bildes 
auf dem Anzeigebildschirm der Anzeigemittel 
bewegt wird. 

12. Vorrichtung zur Steuerung der Position des Bildes 
einer Tonquelle nach Anspruch 9, bei welcher Vor- 
richtung die Virtuell-Lautsprecher-Bildungsmittel 
Steuermittel fur die Position der wenigstens zwei 
virtuellen Lautsprecher (33, 34) enthalten, die vor- 
gegebene Signalverarbeitungen jeweils entspre- 
chend dem Szenenidentifizierungssignal an dem 
Audiosignal so durchfuhren, daG sie die wenigstens 
zwei virtuellen Lautsprecher bilden, von denen das 
Bild der Tonquelle entsprechend dem Audiosignal 
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gebildet wird. 

13. Vorrichtung zur Steuerung der Position des Bildes 
einerTonquelle nach Anspruch 12, bei welcher Vor- 
richtung Audio/Video-lnformationserzeugungsmit- 
tel vorgesehen sind, die ein Panoramas ignai (PS) 
erzeugen, durch das die Position des Bildes der 
Tonquelle an einer bestimmten Unterteilungsstelle 
des linearen Raums zwischen den virtuellen Laut- 
sprechern lokalisiert wird. 



Revendications 



dans lequel un son, correspondant au signal 
sonore recu, genere par les haut-parleurs 
reels, au moins au nombre de deux, est localise 
dans une position situee entre au moins deux 
5 haut-parleurs virtuels, form6s par les moyens, 

au moins au nombre de deux, de formation de 
haut-parleurs virtuels, conformement au rap- 
port de distribution. 

10 2. Appareil de commande de position d'une image so- 
nore selon la revendication 1 , dans lequel le rapport 
de distribution est determine conformement a une 
caracteristique de frequence du signal sonore. 



1 . Appareil de commande de position d'une image so- is 
nore, comportant : 

des moyens (CH 1 0-1 7) d'entree de signaux so- 
nores pour I'entree de signaux sonores ; 
au moins deux haut-parleurs reels (SP(L), SP 20 
(R)) pour delivrer en sortie les signaux sonores 
de sortie ; 

plusieurs moyens (DL10-13, KL10-13, 
KR10-13) de formation de haut-parleurs vir- 
tuels couples aux haut-parleurs reels, chacun 25 
de la plurality de moyens de formation de haut- 
parleurs virtuels traitant un signal sonore qui lui 
est appliqu6 en entree afin qu'un son corres- 
pondant aux signaux sonores traites et gen£re 
par au moins deux des haut-parleurs reels soit 30 
localise dans une position souhaitee, dans un 
espace d'ecoute dans lequel, par consequent, 
chacun de la plurality de moyens de formation 
de haut-parleurs virtuels forme virtuellement un 
haut-parleur virtuel dans la position souhaitee, 35 
chacun de la pluralite de moyens de formation 
de haut-parleurs virtuels generant des signaux 
de cote droit et de cote gauche en traitant le 
signal qui lui est applique en entree ; 
des moyens de melange (AD10-1 3) destines a 40 
melanger les signaux du cote droit delivres en 
sortie par les moyens, au moins au nombre de 
deux, de formation de haut-parleurs virtuels, et 
a appliquer ensuite les signaux melanges du 
cote droit a Tun d'au moins deux des haut- 45 
parleurs reels et a melanger les signaux du cote 
gauche delivrSs en sortie par les moyens, au 
moins au nombre de deux, de formation de 
haut-parleurs virtuels, puis a appliquer les si- 
gnaux melanges du cote gauche a Pautre des so 
haut-parleurs reels, au moins au nombre de 
deux ; 

des moyens de distribution (MTR1 ; 35, 36) 
destines a recevoir un signal sonore des 
moyens d'entree de signaux sonores et a le dis- 55 
tribuer ensuite parmi au moins deux des 
moyens de formation de haut-parleurs virtuels 
selon un rapport de distribution ; 



3. Appareil de commande de position d'une image so- 
nore selon la revendication 1 , comportant en outre 
un circuit de suppression de diaphonie couple aux 
moyens de melange et aux haut-parleurs reels. 

4. Appareil de commande de position d'une image so- 
nore selon la revendication 1, dans lequel les 
moyens d'entree et de signal sonore comprennent 
un generateurde sons commande parun signal MI- 
DI. 

5. Appareil de commande de position d'une image so- 
nore selon la revendication 1, dans lequel les 
moyens de distribution comprennent un dispositif 
de commande matricielle contenant plusieurs mul- 
tiplicateurs et plusieurs additionneurs dans les- 
quels une configuration de connexion est modifiee 
pour s'adapter a une modification d'un protocole de 
distribution de signaux. 

6. Appareil de commande de position d'une image so- 
nore selon la revendication 1 , dans lequel chacun 
des moyens de formation de haut-parleurs virtuels 
comporte : 

un circuit a retard (DL10-13) ayant deux temps 
de retard, le circuit a retard retardant un signal 
sonore qui lui est applique en entree par cha- 
cun des temps de retard afin de delivrer en sor- 
tie deux signaux retardes ; et 
des moyens (KL10-13, KR10-13) duplication 
d'un rapport de repartition destines a appliquer 
un rapport predetermine de repartition aux si- 
gnaux retardes, repartissant ainsi les signaux 
retardes vers les haut-parleurs reels, au moins 
au nombre de deux, en tant que signaux du co- 
te droit et du cote gauche. 

7. Appareil de commande de position d'une image so- 
nore selon la revendication 1, dans lequel la plura- 
lite de moyens de formation de haut-parleurs vir- 
tuels comporte un filtre numSrique a reponse impul- 
sionnelle finie. 
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8. Appareil de commande de position d'une image so- 
nore selon la revendication 1, dans lequel les 
moyens de distribution comprennent des moyens 
de commande de distribution qui commandent le 
taux de distribution pour modifier une position de 
localisation du son entre les positions des haut- 
parleurs virtuels au moins au nombre de deux. 

9. Appareil de commande de position d'une image so- 
nore selon Tune quelconque des revendications 
precedentes, comportant en outre : 

des moyens d'affichage (1 5) destines a afficher 
une image visuelle correspondant a un signal 
video ; 

des moyens (21 ) sur la figure 1 2 de production 
de signaux video, incorpores dans les moyens 
d'entree de signaux sonores, pour produire un 
signal video (VS), un signal sonore (EFS) et un 
signal d'identification de scene (SCS), le signal 
d'identification de scene correspondant a cha- 
que scene d'une image affichee sur les moyens 
d'affichage ; et 

des moyens de commande (31) destines a 
commander au moins deux des moyens (33, 
34) de formation de haut-parleurs virtuels con- 
formement au signal d'identification de scene 
afin que des positions d'au moins deux des 
haut-parleurs virtuels soient commandees en 
reponse au signal d'identification de scene. 

10. Appareil de commande de position d'une image so- 
nore selon la revendication 9, dans lequel les 
moyens (21 ) de production de signaux video pro- 
duisent aussi un signal de panoramiquage (PS) cor- 
respondant au rapport de distribution, puis les 
moyens de commande commandent les moyens de 
distribution (35, 36) conformement au signal de pa- 
noramiquage afin qu'une position du signal sonore 
localisee entre les haut-parleurs virtuels, au moins 
au nombre de deux, soit deplacee en reponse au 
signal de panoramiquage. 

11. Appareil de commande de position d'une image so- 
nore selon I'une quelconque des revendications 1 
a 8, comportant en outre : 

des moyens d'affichage (1 5) destines a afficher 
une image animee predetermined sur un ecran 
d'affichage de ces moyens, ladite image ani- 
m6e correspondant au son devant etre produit 
virtuellement par les haut-parleurs virtuels au 
moins au nombre de deux, une position d'affi- 
chage de ladite image animee correspondant 
a une position d'un son devant etre localise par 
les haut-parleurs virtuels au moins au nombre 
de deux afin que la position du son devant etre 
localisee et correspondant a ladite image ani- 
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mee soit deplacee conformement a un mouve- 
ment de ladite image animee sur I'ecran d'affi- 
chage desdits moyens d'affichage. 

5 12. Appareil de commande de position d'une image so- 
nore selon la revendication 9, dans lequel lesdits 
moyens de formation de haut-parleurs virtuels com- 
prennent au moins deux moyens (33, 34) de com- 
mande de position de haut-parleurs virtuels qui exe- 
10 cutent, respectivement, des traitements de signaux 
predetermines correspondant audit signal d'identi- 
fication de scene sur ledit signal audio afin de for- 
mer au moins deux haut-parleurs virtuels par les- 
quels I'image sonore correspondant audit signal 
audio est formed. 

1 3. Appareil de commande de position d'une image so- 
nore selon la revendication 12, dans lequel des 
moyens de production d'informations audio/video 
sont prevus, produisant un signal de panoramiqua- 
ge (PS) par lequel la position d'une image sonore 
est localised en un certain point de division interieur 
dans un espace lineaire connects entre lesdits 
haut-parleurs virtuels. 
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